{"record":{"id":"e02e9ec09545c9a2","repo":"apache/hadoop","slug":"the-blockcollection-of-is-null-likely-because","errorCode":null,"errorMessage":"The blockCollection of {} is null, likely because the file owning this block was deleted and the block removal is delayed","messagePattern":"The blockCollection of (.+?) is null, likely because the file owning this block was deleted and the block removal is delayed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java","lineNumber":4106,"sourceCode":"          throw new IOException(\"Block (=\" + oldBlock + \") not found\");\n        }\n      }\n      final long oldGenerationStamp = storedBlock.getGenerationStamp();\n      final long oldNumBytes = storedBlock.getNumBytes();\n      //\n      // The implementation of delete operation (see @deleteInternal method)\n      // first removes the file paths from namespace, and delays the removal\n      // of blocks to later time for better performance. When\n      // commitBlockSynchronization (this method) is called in between, the\n      // blockCollection of storedBlock could have been assigned to null by\n      // the delete operation, throw IOException here instead of NPE; if the\n      // file path is already removed from namespace by the delete operation,\n      // throw FileNotFoundException here, so not to proceed to the end of\n      // this method to add a CloseOp to the edit log for an already deleted\n      // file (See HDFS-6825).\n      //\n      if (storedBlock.isDeleted()) {\n        throw new IOException(\"The blockCollection of \" + storedBlock\n            + \" is null, likely because the file owning this block was\"\n            + \" deleted and the block removal is delayed\");\n      }\n      final INodeFile iFile = getBlockCollection(storedBlock);\n      src = iFile.getFullPathName();\n      if (isFileDeleted(iFile)) {\n        throw new FileNotFoundException(\"File not found: \"\n            + src + \", likely due to delayed block removal\");\n      }\n      if ((!iFile.isUnderConstruction() || storedBlock.isComplete()) &&\n          iFile.getLastBlock().isComplete()) {\n        if (LOG.isDebugEnabled()) {\n          LOG.debug(\"Unexpected block (={}) since the file (={}) is not under construction\",\n              oldBlock, iFile.getLocalName());\n        }\n\n        return;\n      }","sourceCodeStart":4088,"sourceCodeEnd":4124,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java#L4088-L4124","documentation":"storedBlock.isDeleted() is true: the block's blockCollection was nulled because the owning file was deleted and physical block removal is delayed for performance. commitBlockSynchronization throws IOException here instead of hitting an NPE, and avoids appending a CloseOp edit for a dead file (HDFS-6825).","triggerScenarios":"A delete removes the path from the namespace first and invalidates blocks later; commitBlockSynchronization arrives inside that window and finds a block whose owner is already detached.","commonSituations":"Users deleting output files at the exact moment a failed write is being recovered; cleanup jobs racing stragglers; committer aborting a task while its pipeline syncs.","solutions":["Treat the write as terminally failed: the file is gone, so no commit is needed","If the file should still exist, verify with getFileStatus and re-run the client operation once block GC has settled","Serialize application-level delete and finalize so they cannot interleave on the same path"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  commitBlockSynchronization(oldBlock, newGs, newLen, ...);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"blockCollection\") && !fs.exists(path)) {\n    return; // file deleted, delayed block removal: no-op\n  }\n  throw e;\n}","preventionTips":["Do not delete files while recovery operations may still be in flight","Treat delayed block removal windows as normal; re-check existence on failure"],"tags":["hdfs","namenode","deleted-file","race-condition","block-gc"],"backgroundTag":"deleted-resource-race","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}