{"record":{"id":"fc94b234cd80560b","repo":"apache/hadoop","slug":"file-directory-path-does-not-exist","errorCode":null,"errorMessage":"File/Directory {path} does not exist","messagePattern":"File/Directory (.+?) does not exist","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java","lineNumber":394,"sourceCode":"        }\n      }\n    } catch (DirectoryIteratorException e) {\n      throw e.getCause();\n    }\n    return list;\n  }\n\n  /**\n   * Ensure that any writes to the given file is written to the storage device\n   * that contains it. This method opens channel on given File and closes it\n   * once the sync is done.<br>\n   * Borrowed from Uwe Schindler in LUCENE-5588\n   * @param fileToSync the file to fsync\n   * @throws IOException raised on errors performing I/O.\n   */\n  public static void fsync(File fileToSync) throws IOException {\n    if (!fileToSync.exists()) {\n      throw new FileNotFoundException(\n          \"File/Directory \" + fileToSync.getAbsolutePath() + \" does not exist\");\n    }\n    boolean isDir = fileToSync.isDirectory();\n\n    // HDFS-13586, FileChannel.open fails with AccessDeniedException\n    // for any directory, ignore.\n    if (isDir && Shell.WINDOWS) {\n      return;\n    }\n\n    // If the file is a directory we have to open read-only, for regular files\n    // we must open r/w for the fsync to have an effect. See\n    // http://blog.httrack.com/blog/2013/11/15/\n    // everything-you-always-wanted-to-know-about-fsync/\n    try(FileChannel channel = FileChannel.open(fileToSync.toPath(),\n        isDir ? StandardOpenOption.READ : StandardOpenOption.WRITE)){\n      fsync(channel, isDir);\n    }","sourceCodeStart":376,"sourceCodeEnd":412,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java#L376-L412","documentation":"Thrown by IOUtils.fsync(File) when the path to sync does not exist on the local filesystem at the time the channel is opened. It is a plain FileNotFoundException: between the caller's creation of the file and the fsync call, the file was deleted, or it was never created. The sync never runs — this is a precondition check, not a sync failure.","triggerScenarios":"Calling IOUtils.fsync(fileToSync) on a path that was removed (e.g. a log segment deleted by a concurrent process) or on a path created only later (ordering bug: fsync before create/flush). Note from the source that on Windows, directory fsync returns early, so this fires for directories only on non-Windows platforms.","commonSituations":"NameNode/edit-log style code where a cleaner thread deletes segments while a writer tries to roll+fsync; checkpoint artifacts deleted by a timeout between write and sync; unit tests on tmp dirs cleaned concurrently; passing a mistyped path that was never written.","solutions":["Check for concurrent deletion of the file (logs, retention/cleanup threads) and serialize delete vs. sync with a lock or lease.","Verify ordering: the file must be created and still open/flushed before fsync is called.","Re-create the file and retry the write+fsync if the artifact is recoverable.","If the path is wrong, fix the path construction — print fileToSync.getAbsolutePath() from the message and compare with the intended location."],"exampleFix":"// before: delete and sync race; FileNotFoundException under load\nif (!file.exists() || !file.isDirectory()) {\n  IOUtils.fsync(file);\n}\n\n// after: hold the lifecycle lock and re-check existence atomically with sync\nsynchronized (fileLifecycleLock) {\n  if (file.exists()) {\n    IOUtils.fsync(file);\n  } else {\n    LOG.warn(\"{} vanished before fsync — skipped\", file.getAbsolutePath());\n  }\n}","handlingStrategy":"validation","validationCode":"synchronized (lifecycleLock) {\n  if (!fileToSync.exists()) {\n    LOG.warn(\"{} deleted before fsync — skipping\", fileToSync);\n    return;\n  }\n  IOUtils.fsync(fileToSync);\n}","typeGuard":null,"tryCatchPattern":"try {\n  IOUtils.fsync(file);\n} catch (FileNotFoundException e) {\n  // file vanished between creation and sync —\n  // re-create and rewrite if recoverable, else skip with a warning\n  handleVanishedFile(file);\n}","preventionTips":["Serialize file deletion (retention/cleaner threads) against fsync with a shared lock or lease.","Ensure create→flush→fsync ordering in code paths; never sync a path you haven't written yet.","On Windows remember directory fsync is a no-op in this method — don't rely on it for durability guarantees there.","Log getAbsolutePath() on failure so the exact vanished path is identifiable."],"tags":["io","fsync","file-not-found","race-condition","hadoop-common"],"backgroundTag":"file-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}