{"record":{"id":"79086a39c7d778da","repo":"apache/hadoop","slug":"no-such-file-or-directory-79086a","errorCode":null,"errorMessage":"No such file or directory","messagePattern":"No such file or directory","errorType":"exception","errorClass":"PathNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/PathData.java","lineNumber":182,"sourceCode":"      inferredSchemeFromPath = checkIfSchemeInferredFromPath(pathString);\n    }\n  }\n\n  // need a static method for the ctor above\n  /**\n   * Get the FileStatus info\n   * @param ignoreFNF if true, stat will be null if the path doesn't exist\n   * @return FileStatus for the given path\n   * @throws IOException if anything goes wrong\n   */\n  private static\n  FileStatus lookupStat(FileSystem fs, String pathString, boolean ignoreFNF)\n  throws IOException {\n    FileStatus status = null;\n    try {\n      status = fs.getFileStatus(new Path(pathString));\n    } catch (FileNotFoundException e) {\n      if (!ignoreFNF) throw new PathNotFoundException(pathString);\n    }\n    // TODO: should consider wrapping other exceptions into Path*Exceptions\n    return status;\n  }\n  \n  private void setStat(FileStatus stat) {\n    this.stat = stat;\n    exists = (stat != null);\n  }\n\n  /**\n   * Updates the paths's file status\n   * @return the updated FileStatus\n   * @throws IOException if anything goes wrong...\n   */\n  public FileStatus refreshStatus() throws IOException {\n    FileStatus status = null;\n    try {","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/PathData.java#L164-L200","documentation":"PathData.lookupStat (PathData.java:182) wraps FileSystem.getFileStatus(): on FileNotFoundException, when the ignoreFNF flag is false it rethrows as PathNotFoundException ('No such file or directory'). The only current caller with ignoreFNF=false is refreshStatus(), so this fires when re-statting a path whose earlier status was already fetched.","triggerScenarios":"Calling PathData.refreshStatus() (or a shell command that calls it, e.g. SetReplication's waitForReplication loop) on a path that was deleted after the PathData was created. TOCTOU: file listed by a glob/expansion, then removed by a concurrent writer/job before the refresh.","commonSituations":"Shell commands processing globs while another process prunes the tree; long-running -setrep -w waits (refreshStatus runs every 10s) where files get deleted mid-wait; tests and tooling that hold PathData objects across mutating operations.","solutions":["Re-verify existence before refreshing: 'if (pd.exists)' is stale — call fs.exists(pd.path) immediately before refreshStatus","Catch PathNotFoundException and treat the path as gone (skip/continue) in iterative processing loops","Avoid concurrent delete-while-processing: coordinate the pipeline so producers finish before consumers stat files"],"exampleFix":"// before\nitem.refreshStatus(); // throws if deleted since enumeration\n\n// after\ntry {\n  item.refreshStatus();\n} catch (PathNotFoundException e) {\n  continue; // vanished concurrently, skip it\n}","handlingStrategy":"try-catch","validationCode":"// existence gate immediately before refresh\nif (!fs.exists(pd.path)) { /* treat as deleted, skip */ }","typeGuard":null,"tryCatchPattern":"catch (PathNotFoundException e) { mark item as vanished and continue the batch; never propagate from a refresh loop }","preventionTips":["Re-stat immediately before use, not from cached enumeration","Design batch loops to tolerate mid-flight deletion","Avoid concurrent delete during -setrep -w waits"],"tags":["file-not-found","race-condition","pathdata","status-refresh","hadoop-shell"],"backgroundTag":"file-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}