apache/hadoop · error · FileAlreadyExistsException

Not a directory: {parent}

Error message

Not a directory: {parent}

What it means

Thrown by AliyunOSSFileSystem.createNonRecursive(): it stats the parent of the target path and requires it to be an existing directory. If getFileStatus(parent) succeeds but reports a file, the parent cannot hold children in OSS's flat key space, so it fails fast with FileAlreadyExistsException("Not a directory: " + parent).

Source

Thrown at hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java:167

  /**
   * {@inheritDoc}
   * @throws FileNotFoundException if the parent directory is not present -or
   * is not a directory.
   */
  @Override
  public FSDataOutputStream createNonRecursive(Path path,
      FsPermission permission,
      EnumSet<CreateFlag> flags,
      int bufferSize,
      short replication,
      long blockSize,
      Progressable progress) throws IOException {
    Path parent = path.getParent();
    if (parent != null) {
      // expect this to raise an exception if there is no parent
      if (!getFileStatus(parent).isDirectory()) {
        throw new FileAlreadyExistsException("Not a directory: " + parent);
      }
    }
    return create(path, permission, flags.contains(CreateFlag.OVERWRITE),
        bufferSize, replication, blockSize, progress);
  }

  @Override
  public boolean delete(Path path, boolean recursive) throws IOException {
    try {
      return innerDelete(getFileStatus(path), recursive);
    } catch (FileNotFoundException e) {
      LOG.debug("Couldn't delete {} - does not exist", path);
      return false;
    }
  }

  /**
   * Delete an object. See {@link #delete(Path, boolean)}.

View on GitHub (pinned to 2add963021)

Solutions

1. Find and remove or rename the file occupying the parent path (e.g., oss://bucket/a) so the directory structure can exist
2. Fix the writer to use consistent path layout so a given name is only ever a file or a directory, never both
3. Use ls/getFileStatus on the parent before createNonRecursive to give a clearer application-level error

Example fix

// before
fs.createNonRecursive(new Path("oss://bucket/a/b"), perm, flags, buf, rep, block, null);
// 'a' exists as a file

// after
// remove or relocate object 'a', then create the directory first
fs.mkdirs(new Path("oss://bucket/a"));
fs.createNonRecursive(new Path("oss://bucket/a/b"), perm, flags, buf, rep, block, null);

Path parent = path.getParent();
if (!fs.exists(parent) || !fs.getFileStatus(parent).isDirectory()) {
  throw new IOException("Parent missing or not a directory: " + parent);
}
fs.createNonRecursive(path, perm, flags, buf, rep, block, null);

catch (FileAlreadyExistsException e) { /* parent is a file: surface which ancestor conflicts and fail */ throw e; }

When it happens

Trigger: Calling createNonRecursive() where some ancestor key exists as a file object — e.g., creating oss://bucket/a/b when object 'a' exists as a file; mixing file and directory usage of the same name in a key hierarchy.

Common situations: Data lakes where an earlier job wrote a file at a path now used as a partition directory; malformed generated paths (a file name reused as a directory prefix); copying HDFS trees that contained file/dir name collisions normalized differently.

Related errors

• {path} is a directory
• Path is a file: {path}
• Failed to rename %s to %s, %s is a file
• Parent path is not a directory: " + parent
• Parent path is not a directory: {} {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4523144413cfea20. Report an issue: GitHub.