apache/hadoop · error · ParentNotDirectoryException

destination parent [{}] is not a directory

Error message

destination parent [{}] is not a directory

What it means

checkDestinationParent stats the parent of the rename destination after the destination itself was found missing. A parent that exists but is not a directory makes the rename invalid, so it throws Hadoop's ParentNotDirectoryException('destination parent [<parent>] is not a directory'). The check only runs when the parent key is non-empty, so the bucket root as parent is accepted.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSObjectBucketUtils.java:195

    if (src.getParent() != dst.getParent()) {
      // deleteUnnecessaryFakeDirectories(dst.getParent());
      createFakeDirectoryIfNecessary(owner, src.getParent());
    }

    return true;
  }

  private static void checkDestinationParent(final OBSFileSystem owner,
      final Path src,
      final Path dst) throws IOException {
    Path parent = dst.getParent();
    if (!OBSCommonUtils.pathToKey(owner, parent).isEmpty()) {
      try {
        FileStatus dstParentStatus = owner.getFileStatus(
            dst.getParent());
        if (!dstParentStatus.isDirectory()) {
          throw new ParentNotDirectoryException(
              "destination parent [" + dst.getParent()
                  + "] is not a directory");
        }
      } catch (FileNotFoundException e2) {
        throw new RenameFailedException(src, dst,
            "destination has no parent ");
      }
    }
  }

  /**
   * Implement rename file.
   *
   * @param owner     OBS File System instance
   * @param srcKey    source object key
   * @param dstKey    destination object key
   * @param srcStatus source object status
   * @throws IOException any problem with rename operation

View on GitHub (pinned to 2add963021)

Solutions

1. Stat the destination parent with getFileStatus and require isDirectory() before renaming
2. Delete or relocate the file that occupies the parent path, then recreate the directory
3. Call fs.mkdirs(dst.getParent()) first; it fails fast if a file blocks any level of the chain

Example fix

// before
fs.rename(src, new Path("/marker/out")); // /marker is a file

// after
Path parent = new Path("/marker");
if (fs.exists(parent) && !fs.getFileStatus(parent).isDirectory()) {
  fs.delete(parent, false);
  fs.mkdirs(parent);
}
fs.rename(src, new Path("/marker/out"));

Path parent = dst.getParent();
if (parent != null && !parent.isRoot() && fs.exists(parent)
    && !fs.getFileStatus(parent).isDirectory()) {
  throw new ParentNotDirectoryException("Destination parent is a file: " + parent);
}

static boolean parentIsDirectory(FileSystem fs, Path p) throws IOException {
  Path parent = p.getParent();
  return parent == null || parent.isRoot()
      || !fs.exists(parent) || fs.getFileStatus(parent).isDirectory();
}

try {
  fs.rename(src, dst);
} catch (ParentNotDirectoryException e) {
  // a file occupies an ancestor of dst: remove it or pick another layout
}

When it happens

Trigger: fs.rename(src, '/plainfile/out') where /plainfile is a regular file; writing output under a path component that was earlier created as a file (marker file, empty success file); partition layouts where a partition value exists as a file.

Common situations: A job or user created a zero-byte marker at the same name later used as a directory; mixed workloads sharing a prefix with inconsistent conventions; tools migrating data that flatten some components into files.

Related errors

• destination has no parent
• {} is not a directory
• {} is root directory
• new destination is an existed directory
• new destination is an existed file

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4dcc9e81cee3ee72. Report an issue: GitHub.