apache/hadoop · error · FileAlreadyExistsException

Not a directory: {}

Error message

Not a directory: {}

What it means

OBSFileSystem.createNonRecursive resolves the parent path and requires it to be an existing directory: if parent != null and !getFileStatus(parent).isDirectory(), it throws FileAlreadyExistsException('Not a directory: ' + parent). Note getFileStatus on a missing parent throws FileNotFoundException first, so this specific message fires when the parent exists as a FILE. The non-recursive contract demands the parent already exist as a directory; the connector does not create intermediate directories.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSFileSystem.java:856

   * @param replication required block replication for the file
   * @param blkSize     block size
   * @param progress    the progress reporter
   * @throws IOException IO failure
   */
  @Override
  public FSDataOutputStream createNonRecursive(
      final Path path,
      final FsPermission permission,
      final EnumSet<CreateFlag> flags,
      final int bufferSize,
      final short replication,
      final long blkSize,
      final Progressable progress)
      throws IOException {
    Path parent = path.getParent();
    if (parent != null && !getFileStatus(parent).isDirectory()) {
      // expect this to raise an exception if there is no parent
      throw new FileAlreadyExistsException("Not a directory: " + parent);
    }
    return create(
        path,
        permission,
        flags.contains(CreateFlag.OVERWRITE),
        bufferSize,
        replication,
        blkSize,
        progress);
  }

  /**
   * Append to an existing file (optional operation).
   *
   * @param f          the existing file to be appended
   * @param bufferSize the size of the buffer to be used
   * @param progress   for reporting progress if it is not null
   * @throws IOException indicating that append is not supported

View on GitHub (pinned to 2add963021)

Solutions

1. Ensure parent directories exist before the call: if (!fs.exists(parent)) fs.mkdirs(parent)
2. If the parent is a stray file, delete or rename it first: fs.delete(parent, false) then mkdirs
3. Check the layout invariant before the job: no path may be used both as a file key and as a directory prefix
4. Prefer the recursive create(...) when intermediate directory creation is acceptable

Example fix

// before
fs.createNonRecursive(new Path("/a/b.txt"), perm, flags, bufSize, repl, blkSize, null);
// /a exists as a FILE -> FileAlreadyExistsException: Not a directory: /a

// after
Path parent = new Path("/a");
if (fs.exists(parent) && !fs.getFileStatus(parent).isDirectory()) {
  fs.delete(parent, false);
}
if (!fs.exists(parent)) {
  fs.mkdirs(parent);
}
fs.createNonRecursive(new Path(parent, "b.txt"), perm, flags, bufSize, repl, blkSize, null);

Path parent = path.getParent();
if (parent != null) {
  if (fs.exists(parent)) {
    if (!fs.getFileStatus(parent).isDirectory()) {
      throw new FileAlreadyExistsException("parent occupied by a file: " + parent);
    }
  } else {
    fs.mkdirs(parent);
  }
}
fs.createNonRecursive(path, ...);

try {
  fs.createNonRecursive(path, ...);
} catch (FileAlreadyExistsException e) {
  if (String.valueOf(e.getMessage()).startsWith("Not a directory")) {
    // parent is a file: relocate or delete it, mkdirs, then retry
  }
  throw e;
}

When it happens

Trigger: Calling createNonRecursive where the parent path is occupied by a regular object (e.g. /a is a file, creating /a/b.txt); file/leaf names colliding with existing object keys used as parents; frameworks using non-recursive create (e.g. some output committers, HBase) against paths whose parents were never mkdir'd or were written as files.

Common situations: Nested output layouts where an earlier step wrote a file at what later becomes a directory level ('file blocks directory' pattern); race between a writer creating /a as a file and another writing /a/b; porting recursive-create code to non-recursive without pre-creating parents.

Related errors

• {} is a directory
• Can't open {} because it is a directory
• {} already exists
• Mkdirs failed to create " + parent.toString()
• {} already exists

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0d4b2d1ebb302721. Report an issue: GitHub.