apache/hadoop · error · IOException

Block (={}) not found

Error message

Block (={}) not found

What it means

commitBlockSynchronization could not find storedBlock for the old block in the NameNode blocks map. With deleteblock=true this is treated as a benign retry (debug log, no-op return); with deleteblock=false it throws IOException because there is no block state left to synchronize.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:4088

    boolean copyTruncate = false;
    BlockInfo truncatedBlock = null;
    try {
      checkOperation(OperationCategory.WRITE);
      // If a DN tries to commit to the standby, the recovery will
      // fail, and the next retry will succeed on the new NN.
  
      checkNameNodeSafeMode(
          "Cannot commitBlockSynchronization while in safe mode");
      final BlockInfo storedBlock = getStoredBlock(
          ExtendedBlock.getLocalBlock(oldBlock));
      if (storedBlock == null) {
        if (deleteblock) {
          // This may be a retry attempt so ignore the failure
          // to locate the block.
          LOG.debug("Block (={}) not found", oldBlock);
          return;
        } else {
          throw new IOException("Block (=" + oldBlock + ") not found");
        }
      }
      final long oldGenerationStamp = storedBlock.getGenerationStamp();
      final long oldNumBytes = storedBlock.getNumBytes();
      //
      // The implementation of delete operation (see @deleteInternal method)
      // first removes the file paths from namespace, and delays the removal
      // of blocks to later time for better performance. When
      // commitBlockSynchronization (this method) is called in between, the
      // blockCollection of storedBlock could have been assigned to null by
      // the delete operation, throw IOException here instead of NPE; if the
      // file path is already removed from namespace by the delete operation,
      // throw FileNotFoundException here, so not to proceed to the end of
      // this method to add a CloseOp to the edit log for an already deleted
      // file (See HDFS-6825).
      //
      if (storedBlock.isDeleted()) {
        throw new IOException("The blockCollection of " + storedBlock

View on GitHub (pinned to 2add963021)

Solutions

1. Check whether the file still exists; if it was deleted, swallow the error (nothing to commit)
2. Otherwise re-fetch current block info via getBlockLocations/LocatedBlock and redo the update with fresh block identity and GS
3. Verify the block pool id in the ExtendedBlock matches the namenode being called

if (!fs.exists(new Path(src))) {
  return; // file gone: nothing to commit
}
LocatedBlocks lbs = dfs.getClient().getLocatedBlocks(path, 0);
// verify oldBlock id still present in lbs before syncing

try {
  commitBlockSynchronization(oldBlock, newGs, newLen, ...);
} catch (IOException e) {
  if (e.getMessage().contains("Block (") && e.getMessage().contains(") not found")
      && !fs.exists(path)) {
    return; // benign: file deleted, block removed
  }
  throw e;
}

When it happens

Trigger: Client retries commitBlockSynchronization after the block was already removed: the owning file was deleted, a previous recovery invalidated the block, or the ExtendedBlock carries the wrong block pool id.

Common situations: File deleted while pipeline recovery was in flight; retry of a failed pipeline setup after another client's lease recovery already reorganized the last block; multi-cluster configs mixing block pool ids.

Related errors

• {} does not exist.
• {}. Name node is in safe mode. {} NamenodeHostName:{}
• The recovery id {} does not match current recovery id {} for
• Unexpected BlockUCState: {} is {} but not UNDER_CONSTRUCTION
• Lease mismatch: {} is accessed by a non lease holder {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e8ed9027f2e49dfc. Report an issue: GitHub.