{"record":{"id":"aecb45891fec6f8b","repo":"apache/hadoop","slug":"replica-generation-stamp-block-generation-stamp","errorCode":null,"errorMessage":"Replica generation stamp < block generation stamp, block={block}, replica={replica}","messagePattern":"Replica generation stamp < block generation stamp, block=(.+?), replica=(.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java","lineNumber":3440,"sourceCode":"        }\n      }\n    }\n  }\n  \n  @Override // FsDatasetSpi\n  public BlockLocalPathInfo getBlockLocalPathInfo(ExtendedBlock block)\n      throws IOException {\n    try (AutoCloseableLock lock = lockManager.readLock(LockLevel.DIR,\n        block.getBlockPoolId(), getStorageUuidForLock(block),\n        datasetSubLockStrategy.blockIdToSubLock(block.getBlockId()))) {\n      final Replica replica = volumeMap.get(block.getBlockPoolId(),\n          block.getBlockId());\n      if (replica == null) {\n        throw new ReplicaNotFoundException(block);\n      }\n      synchronized(replica) {\n        if (replica.getGenerationStamp() < block.getGenerationStamp()) {\n          throw new IOException(\n              \"Replica generation stamp < block generation stamp, block=\"\n                  + block + \", replica=\" + replica);\n        } else if (replica.getGenerationStamp() > block.getGenerationStamp()) {\n          block.setGenerationStamp(replica.getGenerationStamp());\n        }\n      }\n    }\n\n    ReplicaInfo r = getBlockReplica(block);\n    File blockFile = new File(r.getBlockURI());\n    File metaFile = new File(r.getMetadataURI());\n    BlockLocalPathInfo info = new BlockLocalPathInfo(block,\n        blockFile.getAbsolutePath(), metaFile.toString());\n    return info;\n  }\n\n  @Override\n  public void enableTrash(String bpid) {","sourceCodeStart":3422,"sourceCodeEnd":3458,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java#L3422-L3458","documentation":"Same staleness rule as getReplicaVisibleLength, applied on the short-circuit path: getBlockLocalPathInfo refuses a local read when the replica's generation stamp is lower than the stamp in the client's ExtendedBlock. Serving bytes from the old generation to a client holding the new block identity would expose stale data.","triggerScenarios":"Short-circuit read racing lease recovery or truncate commit on the same block; a RegionServer using cached block locations from before a recovery; the replica on this DN missed the generation-stamp bump.","commonSituations":"HBase short-circuit reads immediately after lease recovery; transient after truncate; a DN whose replica never converged to the committed stamp.","solutions":["Retry the read: the client refreshes located blocks and picks a current replica, falling back to remote reads if needed.","Run 'hdfs fsck' to see which replicas lag in generation stamp; the NN heals them via recovery or invalidation.","Restart the lagging DataNode if its replica never converges.","Upgrade client/server combinations with stale block-location caching fixes."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"Replica r = fsDataset.getReplica(block.getBlockPoolId(), block.getBlockId());\nif (r == null || r.getGenerationStamp() < block.getGenerationStamp()) {\n  // stale replica: use a different replica or read remotely\n}","typeGuard":null,"tryCatchPattern":"try {\n  BlockLocalPathInfo info = data.getBlockLocalPathInfo(block);\n} catch (IOException e) {\n  // refresh block locations and fall back to a remote read\n}","preventionTips":["Refresh cached block locations after lease recovery or truncate before short-circuit reads.","Keep replicas converged: investigate replicas that lag in generation stamp via 'hdfs fsck'.","Upgrade clients that cache block locations across recoveries."],"tags":["hdfs","datanode","short-circuit-read","generation-stamp","stale-replica"],"backgroundTag":"hdfs-stale-replica-generation-stamp","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}