{"record":{"id":"b1e8ee9e3f5c4cb8","repo":"apache/hadoop","slug":"file-not-found-likely-due-to-delayed-block-re","errorCode":null,"errorMessage":"File not found: {}, likely due to delayed block removal","messagePattern":"File not found: (.+?), likely due to delayed block removal","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java","lineNumber":4113,"sourceCode":"      // first removes the file paths from namespace, and delays the removal\n      // of blocks to later time for better performance. When\n      // commitBlockSynchronization (this method) is called in between, the\n      // blockCollection of storedBlock could have been assigned to null by\n      // the delete operation, throw IOException here instead of NPE; if the\n      // file path is already removed from namespace by the delete operation,\n      // throw FileNotFoundException here, so not to proceed to the end of\n      // this method to add a CloseOp to the edit log for an already deleted\n      // file (See HDFS-6825).\n      //\n      if (storedBlock.isDeleted()) {\n        throw new IOException(\"The blockCollection of \" + storedBlock\n            + \" is null, likely because the file owning this block was\"\n            + \" deleted and the block removal is delayed\");\n      }\n      final INodeFile iFile = getBlockCollection(storedBlock);\n      src = iFile.getFullPathName();\n      if (isFileDeleted(iFile)) {\n        throw new FileNotFoundException(\"File not found: \"\n            + src + \", likely due to delayed block removal\");\n      }\n      if ((!iFile.isUnderConstruction() || storedBlock.isComplete()) &&\n          iFile.getLastBlock().isComplete()) {\n        if (LOG.isDebugEnabled()) {\n          LOG.debug(\"Unexpected block (={}) since the file (={}) is not under construction\",\n              oldBlock, iFile.getLocalName());\n        }\n\n        return;\n      }\n\n      truncatedBlock = iFile.getLastBlock();\n      final long recoveryId = truncatedBlock.getUnderConstructionFeature()\n          .getBlockRecoveryId();\n      copyTruncate = truncatedBlock.getBlockId() != storedBlock.getBlockId();\n      if(recoveryId != newgenerationstamp) {\n        throw new IOException(\"The recovery id \" + newgenerationstamp","sourceCodeStart":4095,"sourceCodeEnd":4131,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java#L4095-L4131","documentation":"In commitBlockSynchronization, the block's inode resolves but isFileDeleted reports the file's path was removed (or snapshot-deleted) while its blocks await delayed removal. FileNotFoundException is thrown so no CloseOp edit is logged for an already-deleted file (HDFS-6825 family).","triggerScenarios":"Same delete-vs-commit race as the sibling checks, but reached one step later: the block still has its inode attached while the path is already gone.","commonSituations":"File deleted concurrently with block-synchronization retries; snapshot deletions pinning the inode after the path is removed.","solutions":["Confirm with getFileStatus whether the path is really gone; if yes, abandon the write","Retry the client operation only after confirming the file survives and the delete storm is over","Move deletes and finalize/close into the same serialized section of the application"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"for (String p : srcs) {\n  if (!fs.exists(new Path(p))) { /* skip or fail early with clear cause */ }\n}","typeGuard":null,"tryCatchPattern":"try {\n  commitBlockSynchronization(oldBlock, newGs, newLen, ...);\n} catch (FileNotFoundException e) {\n  if (e.getMessage().contains(\"delayed block removal\")) { return; }\n  throw e;\n}","preventionTips":["Re-check getFileStatus when commit-time failures mention delayed removal","Coordinate deletes with writer lifecycle to avoid the race window"],"tags":["hdfs","namenode","file-not-found","race-condition","delayed-block-removal"],"backgroundTag":"file-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}