{"record":{"id":"82d10363942e0868","repo":"apache/hadoop","slug":"the-recovery-id-does-not-match-current-recovery","errorCode":null,"errorMessage":"The recovery id {} does not match current recovery id {} for block {}","messagePattern":"The recovery id (.+?) does not match current recovery id (.+?) for block (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java","lineNumber":4131,"sourceCode":"        throw new FileNotFoundException(\"File not found: \"\n            + src + \", likely due to delayed block removal\");\n      }\n      if ((!iFile.isUnderConstruction() || storedBlock.isComplete()) &&\n          iFile.getLastBlock().isComplete()) {\n        if (LOG.isDebugEnabled()) {\n          LOG.debug(\"Unexpected block (={}) since the file (={}) is not under construction\",\n              oldBlock, iFile.getLocalName());\n        }\n\n        return;\n      }\n\n      truncatedBlock = iFile.getLastBlock();\n      final long recoveryId = truncatedBlock.getUnderConstructionFeature()\n          .getBlockRecoveryId();\n      copyTruncate = truncatedBlock.getBlockId() != storedBlock.getBlockId();\n      if(recoveryId != newgenerationstamp) {\n        throw new IOException(\"The recovery id \" + newgenerationstamp\n                              + \" does not match current recovery id \"\n                              + recoveryId + \" for block \" + oldBlock);\n      }\n\n      if (deleteblock) {\n        Block blockToDel = ExtendedBlock.getLocalBlock(oldBlock);\n        boolean remove = iFile.removeLastBlock(blockToDel) != null;\n        if (remove) {\n          blockManager.removeBlock(storedBlock);\n          FSDirWriteFileOp.persistBlocks(dir, src, iFile, false);\n        }\n      } else {\n        // update last block\n        if(!copyTruncate) {\n          storedBlock.setGenerationStamp(newgenerationstamp);\n          storedBlock.setNumBytes(newlength);\n        }\n","sourceCodeStart":4113,"sourceCodeEnd":4149,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java#L4113-L4149","documentation":"commitBlockSynchronization requires newgenerationstamp to equal the recovery id currently stored in the last block's BlockUnderConstructionFeature. A mismatch means the caller is committing a recovery that has been superseded: another recovery already advanced the block's recovery id.","triggerScenarios":"Two recoveries of the same block race: a client retry after timeout when the first attempt actually succeeded, or another client/lease recovery bumped the id in between; stale client resuming with an old recovery id after NameNode failover.","commonSituations":"Timeout-and-retry logic around pipeline recovery without verifying current state; multiple clients triggering recoverLease concurrently; failover replays in HA deployments.","solutions":["Re-fetch the current block (getBlockLocations / getExtendedBlock) and redo recovery from the fresh generation stamp","Make recovery retries idempotent: compare the block's current GS before re-committing","Ensure only one actor drives recovery for a file at a time"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  commitBlockSynchronization(oldBlock, newGs, newLen, ...);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"does not match current recovery id\")) {\n    LocatedBlock lb = dfs.getClient().getLastLocatedBlock(path); // refresh\n    retryWithCurrentRecoveryId(lb); // idempotent re-commit\n  } else { throw e; }\n}","preventionTips":["Always re-read current block state before re-committing a recovery","One recovery driver per file; avoid concurrent recoverLease callers","Remember a timed-out commit may still have succeeded"],"tags":["hdfs","namenode","block-recovery","generation-stamp","retry"],"backgroundTag":"optimistic-concurrency-conflict","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}