apache/hadoop · error · ParentNotDirectoryException
parent is not a dir
Error message
parent is not a dir
What it means
The second precondition in primitiveMkdir(createParent=false): the parent path exists but is a regular file, so stat.isDirectory() is false and ParentNotDirectoryException('parent is not a dir') is thrown. A file cannot act as a directory, so creating f under it is impossible.
Solutions
- If the file is disposable, delete it (fs.delete(parent, true)) and recreate the directory
- Otherwise choose a different destination layout so files and directories never share prefixes
- Add a pre-flight check that every prefix of f is either absent or a directory
Example fix
// before
fs.primitiveMkdir(new Path("/out/part-0/data"), perm, false); // /out/part-0 is a FILE
// after
Path parent = new Path("/out/part-0/data").getParent();
if (fs.exists(parent) && !fs.getFileStatus(parent).isDirectory()) {
fs.delete(parent, true);
}
fs.mkdirs(parent); Defensive patterns
Strategy: validation
Validate before calling
Path parent = f.getParent();
if (fs.exists(parent) && !fs.getFileStatus(parent).isDirectory()) {
throw new IllegalStateException("Path prefix occupied by a file: " + parent);
}
fs.primitiveMkdir(f, perm, false); Prevention
- Design output layouts where directory prefixes are stable
- Clean stale outputs before creating directory trees
- Assert parent type (directory or absent) in publish preconditions
When it happens
Trigger: primitiveMkdir(f, perm, false) where f.getParent() resolves to an existing file - e.g. creating /a/b/c while /a/b is a file.
Common situations: Stale output files occupying a directory prefix from earlier runs; layouts that changed between file-at-path and directory-at-path; partially cleaned temp trees.
Related errors
AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22).
Data as JSON: /api/errors/591728943e7e861c.
Report an issue: GitHub.
Appendix: source
Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:1405
* @param f the path.
* @param absolutePermission permission.
* @param createParent create parent.
* @throws IOException IO failure.
*/
@Deprecated
protected void primitiveMkdir(Path f, FsPermission absolutePermission,
boolean createParent)
throws IOException {
if (!createParent) { // parent must exist.
// since the this.mkdirs makes parent dirs automatically
// we must throw exception if parent does not exist.
final FileStatus stat = getFileStatus(f.getParent());
if (stat == null) {
throw new FileNotFoundException("Missing parent:" + f);
}
if (!stat.isDirectory()) {
throw new ParentNotDirectoryException("parent is not a dir");
}
// parent does exist - go ahead with mkdir of leaf
}
// Default impl is to assume that permissions do not matter and hence
// calling the regular mkdirs is good enough.
// FSs that implement permissions should override this.
if (!this.mkdirs(f, absolutePermission)) {
throw new IOException("mkdir of "+ f + " failed");
}
}
/**
* Opens an FSDataOutputStream at the indicated Path with write-progress
* reporting. Same as create(), except fails if parent directory doesn't
* already exist.
* @param f the file name to open
* @param overwrite if a file with this name already exists, then if true,
* the file will be overwritten, and if false an error will be thrown.View on GitHub (pinned to 2add963021)