apache/hadoop · error · IOException

The recovery id {} does not match current recovery id {} for

Error message

The recovery id {} does not match current recovery id {} for block {}

What it means

commitBlockSynchronization requires newgenerationstamp to equal the recovery id currently stored in the last block's BlockUnderConstructionFeature. A mismatch means the caller is committing a recovery that has been superseded: another recovery already advanced the block's recovery id.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:4131

        throw new FileNotFoundException("File not found: "
            + src + ", likely due to delayed block removal");
      }
      if ((!iFile.isUnderConstruction() || storedBlock.isComplete()) &&
          iFile.getLastBlock().isComplete()) {
        if (LOG.isDebugEnabled()) {
          LOG.debug("Unexpected block (={}) since the file (={}) is not under construction",
              oldBlock, iFile.getLocalName());
        }

        return;
      }

      truncatedBlock = iFile.getLastBlock();
      final long recoveryId = truncatedBlock.getUnderConstructionFeature()
          .getBlockRecoveryId();
      copyTruncate = truncatedBlock.getBlockId() != storedBlock.getBlockId();
      if(recoveryId != newgenerationstamp) {
        throw new IOException("The recovery id " + newgenerationstamp
                              + " does not match current recovery id "
                              + recoveryId + " for block " + oldBlock);
      }

      if (deleteblock) {
        Block blockToDel = ExtendedBlock.getLocalBlock(oldBlock);
        boolean remove = iFile.removeLastBlock(blockToDel) != null;
        if (remove) {
          blockManager.removeBlock(storedBlock);
          FSDirWriteFileOp.persistBlocks(dir, src, iFile, false);
        }
      } else {
        // update last block
        if(!copyTruncate) {
          storedBlock.setGenerationStamp(newgenerationstamp);
          storedBlock.setNumBytes(newlength);
        }

View on GitHub (pinned to 2add963021)

Solutions

Re-fetch the current block (getBlockLocations / getExtendedBlock) and redo recovery from the fresh generation stamp
Make recovery retries idempotent: compare the block's current GS before re-committing
Ensure only one actor drives recovery for a file at a time

Defensive patterns

Strategy: retry

Try / catch

try {
  commitBlockSynchronization(oldBlock, newGs, newLen, ...);
} catch (IOException e) {
  if (e.getMessage().contains("does not match current recovery id")) {
    LocatedBlock lb = dfs.getClient().getLastLocatedBlock(path); // refresh
    retryWithCurrentRecoveryId(lb); // idempotent re-commit
  } else { throw e; }
}

Prevention

Always re-read current block state before re-committing a recovery
One recovery driver per file; avoid concurrent recoverLease callers
Remember a timed-out commit may still have succeeded

When it happens

Trigger: Two recoveries of the same block race: a client retry after timeout when the first attempt actually succeeded, or another client/lease recovery bumped the id in between; stale client resuming with an old recovery id after NameNode failover.

Common situations: Timeout-and-retry logic around pipeline recovery without verifying current state; multiple clients triggering recoverLease concurrently; failover replays in HA deployments.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/82d10363942e0868. Report an issue: GitHub.