Encountered exception creating directory for heap dumps…

Error message

Encountered exception creating directory for heap dumps, disabling heap dumping.

What it means

This is a warning logged by Beam's fn-harness MemoryMonitor when it fails to create the local directory used to store heap dumps (Files.createDirectories on localDumpFolder throws). When this happens the monitor disables heap dumping entirely (canDumpHeap = false), so on OOM no heap dump will be captured. The message itself wraps the underlying filesystem exception.

Solutions

Verify the configured heap dump folder path does not already exist as a regular file and its parent directories are writable by the harness process user.
Point the heap dump folder to a writable location such as /tmp/heapdumps or a writable mounted volume.
Check the wrapped exception in the log (cause) for the exact filesystem error (access denied vs other) and fix permissions (chmod/chown) accordingly.
If heap dumping is not needed, this warning can be ignored; the pipeline continues without heap dump capture.

Example fix

// before (path is a file or unwritable)
--heapDumpFolder=/var/log
// after (fresh writable directory)
--heapDumpFolder=/tmp/heapdumps

Defensive patterns

Strategy: validation

Validate before calling

File dir = new File(heapDumpFolder);
if (dir.exists() && !dir.isDirectory()) throw new IllegalStateException(heapDumpFolder + " is not a directory");
if (!dir.exists() && !dir.mkdirs() && !dir.setWritable(true)) throw new IllegalStateException("cannot create/write " + heapDumpFolder);

Prevention

Choose a heap dump folder inside a writable location (/tmp) for containerized workers
Do not point the folder at an existing regular file
Run the harness process with a UID that owns or can write the dump directory

When it happens

Trigger: MemoryMonitor.fromOptions is invoked with a heap dump folder (heapDumpFolder option) whose path cannot be created: permission denied, path exists as a regular file, invalid path characters, read-only filesystem, or a security manager blocking file creation.

Common situations: Container images running as non-root with an unwritable dump path (e.g. /tmp not writable), users pointing the dump folder at a mounted volume owned by another UID, or a typo making the path resolve to an existing file.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ea3f2444744988b3. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/harness/src/main/java/org/apache/beam/fn/harness/status/MemoryMonitor.java:222

    SdkHarnessOptions sdkHarnessOptions = options.as(SdkHarnessOptions.class);
    @Nullable String uploadFilePath = sdkHarnessOptions.getRemoteHeapDumpLocation();
    if (uploadFilePath == null) {
      uploadFilePath = options.getTempLocation();
    }
    PortablePipelineOptions portableOptions = options.as(PortablePipelineOptions.class);
    boolean canDumpHeap = false;
    File localDumpFolder = getHeapDumpDir();
    if (portableOptions.getEnableHeapDumps()) {
      if (uploadFilePath == null) {
        LOG.warn(
            "Heap dumps are requested with --enableHeapDumps but neither --remoteHeapDumpLocation nor "
                + "--tempLocation was provided to specify where to copy captured heap dumps to.");
      } else {
        try {
          Files.createDirectories(localDumpFolder.toPath());
          canDumpHeap = true;
        } catch (Exception e) {
          LOG.warn(
              "Encountered exception creating directory for heap dumps, disabling heap dumping.",
              e);
        }
      }
    }

    double gcThrashingPercentagePerPeriod = sdkHarnessOptions.getGCThrashingPercentagePerPeriod();
    return new MemoryMonitor(
        new SystemGCStatsProvider(),
        DEFAULT_SLEEP_TIME_MILLIS,
        DEFAULT_SHUT_DOWN_AFTER_NUM_GCTHRASHING,
        canDumpHeap,
        gcThrashingPercentagePerPeriod,
        uploadFilePath,
        getHeapDumpDir(),
        sdkHarnessOptions.getGzipCompressHeapDumps());
  }

View on GitHub (pinned to 12126d8942)