{"record":{"id":"f006f247bec29b36","repo":"apache/hadoop","slug":"operation-path-is-not-a-directory-its-status-i","errorCode":null,"errorMessage":"{operation}: Path is not a directory; its status is :{status}","messagePattern":"(.+?): Path is not a directory; its status is :(.+?)","errorType":"exception","errorClass":"PathIOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/stages/AbstractJobOrTaskStage.java","lineNumber":602,"sourceCode":"      return path;\n    }\n  }\n\n  /**\n   * Assert that a path is a directory which must exist.\n   * @param operation operation for error reporting.\n   * @param path path path to create.\n   * @return the path\n   * @throws IOException failure\n   * @throws PathIOException mkdirs failed.\n   * @throws FileAlreadyExistsException destination exists.\n   */\n  protected final Path directoryMustExist(\n      final String operation,\n      final Path path) throws IOException {\n    final FileStatus status = getFileStatus(path);\n    if (!status.isDirectory()) {\n      throw new PathIOException(path.toString(),\n          operation\n              + \": Path is not a directory; its status is :\" + status);\n    }\n    return path;\n  }\n\n  /**\n   * Save a task manifest or summary. This will be done by\n   * writing to a temp path and then renaming.\n   * If the destination path exists: Delete it.\n   * This will retry so that a rename failure from abfs load or IO errors\n   * will not fail the task.\n   * @param manifestData the manifest/success file\n   * @param tempPath temp path for the initial save\n   * @param finalPath final path for rename.\n   * @return the manifest saved.\n   * @throws IOException failure to rename after retries.\n   */","sourceCodeStart":584,"sourceCodeEnd":620,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/stages/AbstractJobOrTaskStage.java#L584-L620","documentation":"directoryMustExist() asserts a required path exists AND is a directory; when getFileStatus succeeds but status.isDirectory() is false it throws PathIOException('<operation>: Path is not a directory; its status is :<status>'). Something the committer expects to traverse as a directory (e.g. task-attempt or manifest directories in the output tree) has been replaced by a regular file.","triggerScenarios":"A stage resolving a structural path (manifest/task/job attempt directories) that exists as a file: a user or process created a file at that location, or a partial cleanup converted a directory path into a file.","commonSituations":"Operators or pipelines writing marker/success files into committer-managed directories; output trees shared with other writers; earlier crashed runs leaving a file where a directory belongs.","solutions":["Stat the path shown in the message; delete or relocate the file that occupies the directory location.","Keep external writers (markers, _SUCCESS-style files, downstream readers) out of committer-managed subtrees.","Rerun the job into a clean output directory if the tree's state is uncertain.","If this recurs, audit what process creates files at that path (check file owner/timestamps in the printed status)."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// sanity-check structural paths before the job: required dirs must be dirs\nfor (Path p : requiredDirs) {\n  if (fs.exists(p) && !fs.getFileStatus(p).isDirectory()) {\n    throw new IOException(\"Expected directory but found file at \" + p);\n  }\n}","typeGuard":"static boolean isDirectoryOrAbsent(FileSystem fs, Path p) throws IOException {\n  FileStatus st = fs.exists(p) ? fs.getFileStatus(p) : null;\n  return st == null || st.isDirectory();\n}","tryCatchPattern":null,"preventionTips":["Reserve committer-managed subtrees; write markers and success files only where the committer expects them.","Automate a pre-flight check that structural paths are directories-or-absent.","On collision, remove the file early (before commit) rather than after it fails."],"tags":["hadoop","manifest-committer","path-conflict","filesystem","output-committer"],"backgroundTag":"path-type-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}