{"record":{"id":"4ae4f42c9a1cbf00","repo":"apache/hadoop","slug":"parent-directory-doesn-t-exist","errorCode":null,"errorMessage":"Parent directory doesn't exist: {}","messagePattern":"Parent directory doesn't exist: (.+?)","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java","lineNumber":770,"sourceCode":"    }\n  }\n\n  @Override\n  public FSDataOutputStream create(Path f, FsPermission permission,\n      boolean overwrite, int bufferSize, short replication, long blockSize,\n      Progressable progress) throws IOException {\n    return create(f, permission, overwrite, true, bufferSize,\n        replication, blockSize, progress);\n  }\n\n  private FSDataOutputStream create(Path f, FsPermission permission,\n      boolean overwrite, boolean createParent, int bufferSize,\n      short replication, long blockSize,\n      Progressable progress) throws IOException {\n    Path parent = f.getParent();\n    if (parent != null) {\n      if (!createParent && !exists(parent)) {\n        throw new FileNotFoundException(\"Parent directory doesn't exist: \"\n            + parent);\n      } else if (!mkdirs(parent)) {\n        throw new IOException(\"Mkdirs failed to create \" + parent\n            + \" (exists=\" + exists(parent) + \", cwd=\" + getWorkingDirectory()\n            + \")\");\n      }\n    }\n    final FSDataOutputStream out;\n    if (writeChecksum) {\n      out = new FSDataOutputStream(\n          new ChecksumFSOutputSummer(this, f, overwrite, bufferSize, replication,\n              blockSize, progress, permission), null);\n    } else {\n      out = fs.create(f, permission, overwrite, bufferSize, replication,\n          blockSize, progress);\n      // remove the checksum file since we aren't writing one\n      Path checkFile = getChecksumFile(f);\n      if (fs.exists(checkFile)) {","sourceCodeStart":752,"sourceCodeEnd":788,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L752-L788","documentation":"Inside ChecksumFileSystem.create, when called with createParent=false (which is what createNonRecursive does) and the target's parent directory does not exist, the write is refused with FileNotFoundException naming the missing parent. This is a deliberate guard: 'non-recursive' create promises not to silently mkdir parents, so a missing parent is an error, not something to fix.","triggerScenarios":"Calling fs.createNonRecursive(path, permission, overwrite, ...) on LocalFileSystem (ChecksumFileSystem.java:796 forwards to the private create with createParent=false) when path.getParent() does not exist. Also reachable via FSDataOutputStreamBuilder / FileSystem.create(..., EnumSet<CreateFlag>, ...) variants that route to createNonRecursive.","commonSituations":"Job committers and output writers using createNonRecursive to avoid racing implicit mkdirs (classic MR FileOutputCommitter pattern); a parent directory deleted by a concurrent cleanup job; wrong working directory or misspelled relative path making the resolved parent not exist.","solutions":["Create the parent explicitly before writing: if (!fs.exists(parent)) fs.mkdirs(parent); then call createNonRecursive.","Switch to plain fs.create(...), which passes createParent=true and auto-mkdirs the parent chain.","Print fs.getWorkingDirectory() and the absolute path (the error only names the parent) to catch relative-path/working-directory mistakes."],"exampleFix":"// before\nFSDataOutputStream out = fs.createNonRecursive(new Path(\"out/part-0\"), perms, true, 4096, (short)1, 1<<26, null);\n// throws FileNotFoundException: Parent directory doesn't exist: out\n\n// after\nPath parent = new Path(\"out\");\nif (!fs.exists(parent)) fs.mkdirs(parent);\nFSDataOutputStream out = fs.createNonRecursive(new Path(parent, \"part-0\"), perms, true, 4096, (short)1, 1<<26, null);","handlingStrategy":"validation","validationCode":"Path parent = path.getParent();\nif (parent != null && !fs.exists(parent)) {\n  fs.mkdirs(parent); // or fail with a clear message before createNonRecursive\n}","typeGuard":null,"tryCatchPattern":"try {\n  out = fs.createNonRecursive(path, perms, overwrite, buf, rep, block, null);\n} catch (FileNotFoundException e) {\n  // e.getMessage() names the missing parent; mkdirs and retry once\n}","preventionTips":["Use plain create() when implicit parent creation is acceptable.","With createNonRecursive, always precede it with an explicit exists/mkdirs on the parent.","Log fs.getWorkingDirectory() alongside relative paths to catch resolution surprises."],"tags":["hadoop","filesystem","create","missing-directory","local-filesystem","file-not-found"],"backgroundTag":"missing-parent-directory","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}