{"record":{"id":"1e2b4bd887be78e1","repo":"apache/hadoop","slug":"stagestatisticname-mkdirs-returned-false","errorCode":null,"errorMessage":"{stageStatisticName}: mkdirs() returned false","messagePattern":"(.+?): mkdirs\\(\\) returned false","errorType":"exception","errorClass":"PathIOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/stages/AbstractJobOrTaskStage.java","lineNumber":496,"sourceCode":"        operations.deleteFile(path));\n  }\n\n  /**\n   * Create a directory.\n   * @param path path\n   * @param escalateFailure escalate \"false\" to PathIOE\n   * @return true if the directory was created/exists.\n   * @throws IOException IO Failure.\n   */\n  public final boolean mkdirs(\n      final Path path,\n      final boolean escalateFailure)\n      throws IOException {\n    LOG.trace(\"{}: mkdirs('{}')\", getName(), path);\n    return trackDuration(getIOStatistics(), OP_MKDIRS, () -> {\n      boolean success = operations.mkdirs(path);\n      if (!success && escalateFailure) {\n        throw new PathIOException(path.toUri().toString(),\n            stageStatisticName + \": mkdirs() returned false\");\n      }\n      return success;\n    });\n\n  }\n\n  /**\n   * List all directly files under a path.\n   * Async implementations may under-report their durations.\n   * @param path path\n   * @return iterator over the results.\n   * @throws IOException IO Failure.\n   */\n  protected final RemoteIterator<FileStatus> listStatusIterator(\n      final Path path)\n      throws IOException {\n    LOG.trace(\"{}: listStatusIterator('{}')\", getName(), path);","sourceCodeStart":478,"sourceCodeEnd":514,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/stages/AbstractJobOrTaskStage.java#L478-L514","documentation":"Manifest committer stages create directories through operations.mkdirs() tracked under the OP_MKDIRS statistic. Java's FileSystem.mkdirs() returns false (rather than throwing) when creation fails; when escalateFailure is true the stage converts that false into PathIOException('<stageStatisticName>: mkdirs() returned false'). A false return usually means the target exists as a file, or the caller lacks permission to create it.","triggerScenarios":"Creating a stage directory (job/job-attempt/task attempt paths in the output tree) where: a FILE already exists at that exact path, the parent is not writable, or the filesystem is read-only/unavailable in a way that surfaces as a boolean failure.","commonSituations":"A leftover file occupying a directory path from an earlier failed run; output directories owned by another user; quota or permission changes between job submission and commit; a user or external process having written into the committer's namespace.","solutions":["Stat the failing path (printed in the PathIOException): if it is a file, delete or relocate it and rerun.","Fix ownership/permissions on the output tree so the running user can create directories.","Rerun into a fresh output directory to rule out stale state.","Check the nested filesystem diagnostics/logs around the OP_MKDIRS statistic for the underlying refusal."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// before running the job: every directory path the committer will create\n// must be absent or already a directory, and its parent writable\nFileSystem fs = outputDir.getFileSystem(conf);\nif (fs.exists(outputDir) && !fs.getFileStatus(outputDir).isDirectory()) {\n  throw new IOException(\"Output path is a file, not a directory: \" + outputDir);\n}\nif (fs.exists(outputDir)\n    && !fs.getFileStatus(outputDir).getPermission().getUserAction().implies(FsAction.WRITE)) {\n  throw new IOException(\"Output dir not writable: \" + outputDir);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pre-create output directories owned by the job user, or let the job create them fresh.","Keep files out of committer-managed directory paths.","Treat mkdirs()==false as actionable, not transient: inspect status before retrying."],"tags":["hadoop","manifest-committer","mkdirs","filesystem","permissions"],"backgroundTag":"mkdirs-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}