apache/hadoop · error · IOException

Replica generation stamp < block generation stamp, block={bl

Error message

Replica generation stamp < block generation stamp, block={block}, replica={replica}

What it means

Same staleness rule as getReplicaVisibleLength, applied on the short-circuit path: getBlockLocalPathInfo refuses a local read when the replica's generation stamp is lower than the stamp in the client's ExtendedBlock. Serving bytes from the old generation to a client holding the new block identity would expose stale data.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java:3440

        }
      }
    }
  }
  
  @Override // FsDatasetSpi
  public BlockLocalPathInfo getBlockLocalPathInfo(ExtendedBlock block)
      throws IOException {
    try (AutoCloseableLock lock = lockManager.readLock(LockLevel.DIR,
        block.getBlockPoolId(), getStorageUuidForLock(block),
        datasetSubLockStrategy.blockIdToSubLock(block.getBlockId()))) {
      final Replica replica = volumeMap.get(block.getBlockPoolId(),
          block.getBlockId());
      if (replica == null) {
        throw new ReplicaNotFoundException(block);
      }
      synchronized(replica) {
        if (replica.getGenerationStamp() < block.getGenerationStamp()) {
          throw new IOException(
              "Replica generation stamp < block generation stamp, block="
                  + block + ", replica=" + replica);
        } else if (replica.getGenerationStamp() > block.getGenerationStamp()) {
          block.setGenerationStamp(replica.getGenerationStamp());
        }
      }
    }

    ReplicaInfo r = getBlockReplica(block);
    File blockFile = new File(r.getBlockURI());
    File metaFile = new File(r.getMetadataURI());
    BlockLocalPathInfo info = new BlockLocalPathInfo(block,
        blockFile.getAbsolutePath(), metaFile.toString());
    return info;
  }

  @Override
  public void enableTrash(String bpid) {

View on GitHub (pinned to 2add963021)

Solutions

  1. Retry the read: the client refreshes located blocks and picks a current replica, falling back to remote reads if needed.
  2. Run 'hdfs fsck' to see which replicas lag in generation stamp; the NN heals them via recovery or invalidation.
  3. Restart the lagging DataNode if its replica never converges.
  4. Upgrade client/server combinations with stale block-location caching fixes.
Defensive patterns

Strategy: try-catch

Validate before calling

Replica r = fsDataset.getReplica(block.getBlockPoolId(), block.getBlockId());
if (r == null || r.getGenerationStamp() < block.getGenerationStamp()) {
  // stale replica: use a different replica or read remotely
}

Try / catch

try {
  BlockLocalPathInfo info = data.getBlockLocalPathInfo(block);
} catch (IOException e) {
  // refresh block locations and fall back to a remote read
}

Prevention

When it happens

Trigger: Short-circuit read racing lease recovery or truncate commit on the same block; a RegionServer using cached block locations from before a recovery; the replica on this DN missed the generation-stamp bump.

Common situations: HBase short-circuit reads immediately after lease recovery; transient after truncate; a DN whose replica never converged to the committed stamp.

Related errors


AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/aecb45891fec6f8b. Report an issue: GitHub.