apache/beam · error · IOException

Unable to create parent directories for ''

Error message

Unable to create parent directories for ''

What it means

LocalFileSystem.create attempts to mkdirs the parent directory of the target file before opening an output channel; if creation fails (and the directory still doesn't exist) it throws IOException 'Unable to create parent directories for ...'.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/io/LocalFileSystem.java:119

  @VisibleForTesting
  List<MatchResult> match(String baseDir, List<String> specs) throws IOException {
    ImmutableList.Builder<MatchResult> ret = ImmutableList.builder();
    for (String spec : specs) {
      ret.add(matchOne(baseDir, spec));
    }
    return ret.build();
  }

  @Override
  protected WritableByteChannel create(LocalResourceId resourceId, CreateOptions createOptions)
      throws IOException {
    LOG.debug("creating file {}", resourceId);
    File absoluteFile = resourceId.getPath().toFile().getAbsoluteFile();
    if (absoluteFile.getParentFile() != null
        && !absoluteFile.getParentFile().exists()
        && !absoluteFile.getParentFile().mkdirs()
        && !absoluteFile.getParentFile().exists()) {
      throw new IOException("Unable to create parent directories for '" + resourceId + "'");
    }
    return Channels.newChannel(new BufferedOutputStream(new FileOutputStream(absoluteFile)));
  }

  @Override
  protected ReadableByteChannel open(LocalResourceId resourceId) throws IOException {
    LOG.debug("opening file {}", resourceId);
    @SuppressWarnings("resource") // The caller is responsible for closing the channel.
    FileInputStream inputStream = new FileInputStream(resourceId.getPath().toFile());
    // Use this method for creating the channel (rather than new FileChannel) so that we get
    // regular FileNotFoundException. Closing the underyling channel will close the inputStream.
    return inputStream.getChannel();
  }

  @Override
  protected void copy(List<LocalResourceId> srcResourceIds, List<LocalResourceId> destResourceIds)
      throws IOException {
    checkArgument(

View on GitHub (pinned to 12126d8942)

Solutions

1. Check filesystem permissions on the parent path and fix with chmod/chown or run as a user with write access
2. Verify no regular file exists with the same name as a required parent directory
3. Create the parent directories yourself before calling create and verify success
4. Ensure the mount point is writable inside the container/worker

Example fix

// before
File out = new File("/data/readonly/out.txt"); // parent missing, not writable
try (WritableByteChannel ch = FileSystems.create(LocalResourceId.fromPath(...), "text/plain")) {...}
// after
File parent = out.getParentFile();
if (!parent.exists() && !parent.mkdirs()) {
  throw new IOException("Cannot create " + parent + "; check permissions");
}
try (WritableByteChannel ch = FileSystems.create(...)) {...}

File parent = new File(path).getAbsoluteFile().getParentFile();
if (parent != null && !parent.exists() && !parent.mkdirs()) {
  throw new IOException("Cannot create parent dir: " + parent + "; check permissions/mounts");
}

try (WritableByteChannel ch = FileSystems.create(resourceId, mimeType)) {
  // write
} catch (IOException e) {
  if (e.getMessage().startsWith("Unable to create parent directories")) {
    LOG.error("Fix permissions or create {} manually", new File(path).getParent());
  }
  throw e;
}

When it happens

Trigger: Writing via FileSystems.create(LocalResourceId...) where the parent directory doesn't exist and mkdirs fails — due to missing write permission on the parent, the path component being an existing regular file, or a read-only filesystem.

Common situations: Writing to /tmp-like paths in sandboxed workers where permissions differ; a file exists where a directory is expected (e.g. output prefix collides with an existing file); running in containers with read-only mounts.

Understand the failure class

Background: mkdir permission denied (EACCES): failed to create directory errors explained — this error's family across 32 libraries.

Related errors

• Support for move options is not yet implemented.
• Timing number 0b" + timingNumber.toString(2) + " has more th
• No proto encoding for PaneInfoCoder, always part of Windowed
• Runner does not support draining.
• cannot encode a null Integer

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3846bc55fbfb5130. Report an issue: GitHub.