apache/hadoop · error · IOException
Replica generation stamp < block generation stamp, block={bl
Error message
Replica generation stamp < block generation stamp, block={block}, replica={replica} What it means
Same staleness rule as getReplicaVisibleLength, applied on the short-circuit path: getBlockLocalPathInfo refuses a local read when the replica's generation stamp is lower than the stamp in the client's ExtendedBlock. Serving bytes from the old generation to a client holding the new block identity would expose stale data.
Source
Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java:3440
}
}
}
}
@Override // FsDatasetSpi
public BlockLocalPathInfo getBlockLocalPathInfo(ExtendedBlock block)
throws IOException {
try (AutoCloseableLock lock = lockManager.readLock(LockLevel.DIR,
block.getBlockPoolId(), getStorageUuidForLock(block),
datasetSubLockStrategy.blockIdToSubLock(block.getBlockId()))) {
final Replica replica = volumeMap.get(block.getBlockPoolId(),
block.getBlockId());
if (replica == null) {
throw new ReplicaNotFoundException(block);
}
synchronized(replica) {
if (replica.getGenerationStamp() < block.getGenerationStamp()) {
throw new IOException(
"Replica generation stamp < block generation stamp, block="
+ block + ", replica=" + replica);
} else if (replica.getGenerationStamp() > block.getGenerationStamp()) {
block.setGenerationStamp(replica.getGenerationStamp());
}
}
}
ReplicaInfo r = getBlockReplica(block);
File blockFile = new File(r.getBlockURI());
File metaFile = new File(r.getMetadataURI());
BlockLocalPathInfo info = new BlockLocalPathInfo(block,
blockFile.getAbsolutePath(), metaFile.toString());
return info;
}
@Override
public void enableTrash(String bpid) {View on GitHub (pinned to 2add963021)
Solutions
- Retry the read: the client refreshes located blocks and picks a current replica, falling back to remote reads if needed.
- Run 'hdfs fsck' to see which replicas lag in generation stamp; the NN heals them via recovery or invalidation.
- Restart the lagging DataNode if its replica never converges.
- Upgrade client/server combinations with stale block-location caching fixes.
Defensive patterns
Strategy: try-catch
Validate before calling
Replica r = fsDataset.getReplica(block.getBlockPoolId(), block.getBlockId());
if (r == null || r.getGenerationStamp() < block.getGenerationStamp()) {
// stale replica: use a different replica or read remotely
} Try / catch
try {
BlockLocalPathInfo info = data.getBlockLocalPathInfo(block);
} catch (IOException e) {
// refresh block locations and fall back to a remote read
} Prevention
- Refresh cached block locations after lease recovery or truncate before short-circuit reads.
- Keep replicas converged: investigate replicas that lag in generation stamp via 'hdfs fsck'.
- Upgrade clients that cache block locations across recoveries.
When it happens
Trigger: Short-circuit read racing lease recovery or truncate commit on the same block; a RegionServer using cached block locations from before a recovery; the replica on this DN missed the generation-stamp bump.
Common situations: HBase short-circuit reads immediately after lease recovery; transient after truncate; a DN whose replica never converged to the committed stamp.
Related errors
- replica.getGenerationStamp() < block.getGenerationStamp(), b
- Replica gen stamp < block genstamp, block={block}, replica={
- ProvidedReplica does not yet support writes
- The new recovery id: {} must be greater than the current one
- Cannot append to a replica with unexpected generation stamp
AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22).
Data as JSON: /api/errors/aecb45891fec6f8b.
Report an issue: GitHub.