What causes "Can't make directory for path '%s' since it is a file." in apache/hadoop?

Identify the file named in the message, confirm it is disposable, then delete or rename it (fs.rename to a '_files/' prefix or delete).

apache/hadoop · error · FileAlreadyExistsException

Can't make directory for path '%s' since it is a file.

Error message

Can't make directory for path '%s' since it is a file.

What it means

When innerMkdirs cannot find the requested directory, it walks UP the ancestor chain (fPart = path.getParent(); do/while to root) and stats each parent. If any ancestor exists as a FILE, it throws FileAlreadyExistsException("Can't make directory for path '<ancestor>' since it is a file.") — you cannot create a directory underneath an object key. Note the reported path is the offending ANCESTOR (fPart), not the path you passed, which often confuses readers of the message.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSCommonUtils.java:731

    FileStatus fileStatus;
    try {
      fileStatus = owner.getFileStatus(path);

      if (fileStatus.isDirectory()) {
        return true;
      } else {
        throw new FileAlreadyExistsException("Path is a file: " + path);
      }
    } catch (FileNotFoundException e) {
      Path fPart = path.getParent();
      do {
        try {
          fileStatus = owner.getFileStatus(fPart);
          if (fileStatus.isDirectory()) {
            break;
          }
          if (fileStatus.isFile()) {
            throw new FileAlreadyExistsException(
                String.format("Can't make directory for path '%s'"
                    + " since it is a file.", fPart));
          }
        } catch (FileNotFoundException fnfe) {
          LOG.debug("file {} not fount, but ignore.", path);
        }
        fPart = fPart.getParent();
      } while (fPart != null);

      String key = pathToKey(owner, path);
      if (owner.isFsBucket()) {
        OBSPosixBucketUtils.fsCreateFolder(owner, key);
      } else {
        OBSObjectBucketUtils.createFakeDirectory(owner, key);
      }
      return true;
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

Identify the file named in the message, confirm it is disposable, then delete or rename it (fs.rename to a '_files/' prefix or delete).
Restructure the layout so every path component used as a directory is only ever a directory/prefix.
Add a layout lint at job start: for each mkdirs target, verify no ancestor exists as a file (cheap getFileStatus walk) and fail with an actionable message.
If the file must stay, move your data tree to a sibling prefix that has no file ancestors.

Example fix

// before
fs.mkdirs(new Path("obs://b/a/b/c"));
// throws: Can't make directory for path 'obs://b/a' since it is a file.

// after
Path a = new Path("obs://b/a");
if (fs.getFileStatus(a).isFile()) {
  fs.rename(a, new Path("obs://b/_files/a")); // relocate conflicting object
}
fs.mkdirs(new Path("obs://b/a/b/c"));

Defensive patterns

Strategy: validation

Validate before calling

static void ensureAncestorsAreDirs(FileSystem fs, Path p) throws IOException {
  for (Path a = p.getParent(); a != null && !a.isRoot(); a = a.getParent()) {
    if (fs.exists(a) && fs.getFileStatus(a).isFile()) {
      throw new FileAlreadyExistsException("ancestor is a file: " + a);
    }
  }
  fs.mkdirs(p);
}

Try / catch

try {
  fs.mkdirs(path);
} catch (FileAlreadyExistsException e) {
  if (String.valueOf(e.getMessage()).contains("since it is a file")) {
    // message names the offending ANCESTOR, not 'path'
    Path offender = extractQuotedPath(e.getMessage());
    throw new LayoutConflictException("file blocks directory tree at " + offender, e);
  }
  throw e;
}

Prevention

Never write a plain object at a key that other code uses as a directory prefix.
When adding partition depth to an existing layout, migrate the old leaf objects first.
Remember the message reports the ancestor path — parse it to find the blocker quickly.

When it happens

Trigger: mkdirs('obs://b/a/b/c') when object 'obs://b/a' exists as a file: the walk hits a, sees isFile(), throws naming 'a'. Typical with key layouts where 'a' was written as a plain object and later code assumes it is a directory; flatten-then-hierarchical schema changes; upstream producers writing marker files at directory prefixes.

Common situations: Schema evolution adding partition levels under a formerly leaf key; different writers disagreeing whether a prefix is a file or dir; manual uploads of files named like directories; migration from another FS where such layout was legal.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/097779aee03d6337. Report an issue: GitHub.