apache/hadoop · error · ParentNotDirectoryException

Rename destination parent {parent} is a file.

Error message

Rename destination parent {parent} is a file.

What it means

When the destination does not exist, renameInternal validates its parent: if getFileStatus(dst.getParent()) resolves to a regular file, the destination cannot be created beneath it and ParentNotDirectoryException is thrown before renameInternal runs. This is the same ENOTDIR-style failure the HDFS NameNode reports, surfaced client-side by FileContext.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java:892

      }
      if (!overwrite) {
        throw new FileAlreadyExistsException("Rename destination " + dst
            + " already exists.");
      }
      // Delete the destination that is a file or an empty directory
      if (dstStatus.isDirectory()) {
        RemoteIterator<FileStatus> list = listStatusIterator(dst);
        if (list != null && list.hasNext()) {
          throw new IOException(
              "Rename cannot overwrite non empty destination directory " + dst);
        }
      }
      delete(dst, false);
    } else {
      final Path parent = dst.getParent();
      final FileStatus parentStatus = getFileStatus(parent);
      if (parentStatus.isFile()) {
        throw new ParentNotDirectoryException("Rename destination parent "
            + parent + " is a file.");
      }
    }
    renameInternal(src, dst);
  }
  
  /**
   * Returns true if the file system supports symlinks, false otherwise.
   * @return true if filesystem supports symlinks
   */
  public boolean supportsSymlinks() {
    return false;
  }
  
  /**
   * The specification of this method matches that of  
   * {@link FileContext#createSymlink(Path, Path, boolean)};
   *

View on GitHub (pinned to 2add963021)

Solutions

1. Remove or relocate the conflicting parent file, then fc.mkdir(parent, permission, true) before renaming
2. Adjust destination naming so every parent component is always a directory
3. Validate dst.getParent() with getFileStatus at job startup and fail fast with a clear message

Example fix

// before
fc.rename(src, new Path("/out/part-0/data")); // /out/part-0 is a file

// after
Path parent = dst.getParent();
if (fc.util().exists(parent) && fc.getFileStatus(parent).isFile()) {
  fc.delete(parent, false);
  fc.mkdir(parent, FsPermission.getDirDefault(), true);
}
fc.rename(src, dst);

Path parent = dst.getParent();
if (fc.util().exists(parent)) {
  FileStatus ps = fc.getFileStatus(parent);
  if (!ps.isDirectory()) {
    throw new IllegalStateException("Rename parent is a file: " + parent);
  }
} else {
  fc.mkdir(parent, FsPermission.getDirDefault(), true);
}

catch (ParentNotDirectoryException e) { /* remove/relocate the conflicting parent file, mkdir parent, retry */ }

When it happens

Trigger: fc.rename(f, new Path("/out/part-0/data")) when /out/part-0 is an existing file; dynamically built paths (date/part keys) whose parent component collides with a file written by an earlier flat layout.

Common situations: Output layouts that append extra path components after an upstream schema change; a previous run writing a file exactly where the next run expects a directory; committer working-dir misconfiguration in MapReduce/Spark jobs.

Related errors

• Cannot rename symlink {src} to its target {dst}
• Source {src} and destination {dst} must both be directories
• Rename destination {dst} already exists.
• Rename cannot overwrite non empty destination directory {dst
• File system does not support symlinks

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/aec790598879d976. Report an issue: GitHub.