apache/hadoop · error · IOException

FileSystem does not support non-recursivemkdir

Error message

FileSystem does not support non-recursivemkdir

What it means

mkdirs(Path, FsPermission, createParent) resolves symlinks with FileSystemLinkResolver; when the final path component lands on another filesystem, next(fs, p) runs on that filesystem. The generic FileSystem API has no non-recursive mkdir, so when createParent is false the client cannot preserve the semantics and throws IOException rather than silently creating parents.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java:1606

  private boolean mkdirsInternal(Path f, final FsPermission permission,
      final boolean createParent) throws IOException {
    statistics.incrementWriteOps(1);
    storageStatistics.incrementOpCounter(OpType.MKDIRS);
    Path absF = fixRelativePart(f);
    return new FileSystemLinkResolver<Boolean>() {
      @Override
      public Boolean doCall(final Path p) throws IOException {
        return dfs.mkdirs(getPathName(p), permission, createParent);
      }

      @Override
      public Boolean next(final FileSystem fs, final Path p)
          throws IOException {
        // FileSystem doesn't have a non-recursive mkdir() method
        // Best we can do is error out
        if (!createParent) {
          throw new IOException("FileSystem does not support non-recursive"
              + "mkdir");
        }
        return fs.mkdirs(p, permission);
      }
    }.resolve(this, absF);
  }

  @SuppressWarnings("deprecation")
  @Override
  protected boolean primitiveMkdir(Path f, FsPermission absolutePermission)
      throws IOException {
    statistics.incrementWriteOps(1);
    storageStatistics.incrementOpCounter(OpType.PRIMITIVE_MKDIR);
    return dfs.primitiveMkdir(getPathName(f), absolutePermission);
  }


  @Override

View on GitHub (pinned to 2add963021)

Solutions

1. Call createParent=true (or plain fs.mkdirs(p, perm)) if creating parents on the target filesystem is acceptable.
2. Create the directory on the target filesystem directly: resolve the link target and open the right FileSystem instance.
3. Restructure so mkdirs never traverses a cross-filesystem symlink.
4. Ensure parent directories already exist on the target filesystem so the non-recursive semantics are trivially satisfied.

Example fix

// before
boolean ok = hdfs.mkdirs(linkPath, perm, /*createParent*/ false); // IOException cross-filesystem

// after
Path target = hdfs.getLinkTarget(linkPath);
FileSystem targetFs = FileSystem.get(target.toUri(), conf);
boolean ok = targetFs.mkdirs(target, perm, false);

// Only needed when createParent=false and path may be a symlink:
FileStatus st = fs.getFileLinkStatus(p);
boolean crossFs = st.isSymlink()
    && !fs.getUri().equals(FileSystem.get(st.getSymlink().toUri(), conf).getUri());
if (crossFs && !createParent) { /* use fallback path */ }

try {
  ok = fs.mkdirs(p, perm, false);
} catch (IOException e) {
  if (e.getMessage().contains("non-recursive")) {
    ok = fs.mkdirs(p, perm, true); // or resolve target fs
  } else throw e;
}

When it happens

Trigger: Calling fs.mkdirs(path, perm, createParent=false) where path (after symlink resolution) crosses onto a different FileSystem, e.g. an HDFS symlink pointing to a local or object-store directory.

Common situations: Tooling that creates leaf directories atomically (createParent=false) over data layouts that mix HDFS with other schemes via symlinks; migrations where a mount table entry now resolves cross-scheme; unit environments using viewfs/local mounts.

Related errors

• Link resolution does not work with multiple file systems for
• getFileChecksum(Path, long) is not supported by
• Cannot perform snapshot operations on a symlink to a non-Dis
• Symlinks not supported
• Filesystem does not support symlinks!

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a8bbdb4ef27908c2. Report an issue: GitHub.