What causes "BlockId {} is not valid." in apache/hadoop?

Confirm the block still exists NameNode-side: hdfs fsck /path -files -blocks | grep , and check whether the file was deleted/replaced under the reader.

apache/hadoop · error · FileNotFoundException

BlockId {} is not valid.

Error message

BlockId {} is not valid.

What it means

Thrown as FileNotFoundException from FsDatasetImpl.getBlockReplica(bpid, blockId) when validateBlockFile returns null, i.e. no replica for that blockId exists in the volumeMap for the block pool. It is the lookup primitive behind most read/length/meta operations on a block, so nearly any client read path that reaches the DataNode with a stale or unknown block ID surfaces it.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java:830

  @Override // FsDatasetSpi
  public long getLength(ExtendedBlock b) throws IOException {
    return getBlockReplica(b).getBlockDataLength();
  }

  /**
   * Get File name for a given block.
   */
  private ReplicaInfo getBlockReplica(ExtendedBlock b) throws IOException {
    return getBlockReplica(b.getBlockPoolId(), b.getBlockId());
  }
  
  /**
   * Get File name for a given block.
   */
  ReplicaInfo getBlockReplica(String bpid, long blockId) throws IOException {
    ReplicaInfo r = validateBlockFile(bpid, blockId);
    if (r == null) {
      throw new FileNotFoundException("BlockId " + blockId + " is not valid.");
    }
    return r;
  }

  @Override // FsDatasetSpi
  public InputStream getBlockInputStream(ExtendedBlock b,
      long seekOffset) throws IOException {

    ReplicaInfo info;
    try (AutoCloseableLock lock = lockManager.readLock(LockLevel.DIR,
        b.getBlockPoolId(), getStorageUuidForLock(b),
        datasetSubLockStrategy.blockIdToSubLock(b.getBlockId()))) {
      info = volumeMap.get(b.getBlockPoolId(), b.getLocalBlock());
    }

    if (info != null && info.getVolume().isTransientStorage()) {
      ramDiskReplicaTracker.touch(b.getBlockPoolId(), b.getBlockId());
      datanode.getMetrics().incrRamDiskBlocksReadHits();

View on GitHub (pinned to 2add963021)

Solutions

Confirm the block still exists NameNode-side: hdfs fsck /path -files -blocks | grep <blockId>, and check whether the file was deleted/replaced under the reader.
Verify the block pool ID matches this DataNode/cluster (VERSION, clusterID/blockpoolID) — a mismatched ExtendedBlock always misses.
If a volume failed, let the DataNode re-register and re-report (or restart it) so the NameNode prunes the missing replicas and re-replicates.
Retry the read from the client with a fresh block-location lookup (getBlockLocations) instead of a cached ExtendedBlock.

Defensive patterns

Strategy: try-catch

Validate before calling

// Cheap pre-check before read-style APIs: resolve the replica first.
ReplicaInfo r = fsDataset.getReplica(b.getBlockPoolId(), b.getBlockId());
if (r == null) {
  // refresh block locations instead of reading a dead ExtendedBlock
  locatedBlocks = dfsClient.getLocatedBlocks(path, offset, len);
  return readFrom(locatedBlocks);
}

Try / catch

// Distinguish 'no such replica' from other IO problems; refresh locations on it.
try {
  return fsDataset.getBlockInputStream(b, offset);
} catch (FileNotFoundException e) { // getBlockReplica signals via FNFE
  LOG.warn("Block {} absent on this DN, refreshing locations", b);
  return readFromFreshLocations(dfsClient, b);
}

Prevention

Never cache ExtendedBlock/locations across delete-adjacent operations; re-fetch getBlockLocations after file mutations.
Check replica presence via FsDatasetSpi.getReplica(bpid, blkId) before volume-local operations.
Keep DataNode block reports fresh (triggerBlockReport) in test harnesses that mutate blocks directly.

When it happens

Trigger: Calling getBlockInputStream/getMetaDataInputStream/validateBlock-style flows (or getBlockReplica directly) for a (blockPoolId, blockId) that has no entry in volumeMap: block already deleted (NameNode tells a slow client to read it), block invalidated between report cycles, wrong block pool ID, or replica lost with a volume failure.

Common situations: Client retries a read after the block was deleted/replaced (lease recovery, truncation); DataNode just recovered from a disk loss so its volumeMap no longer has replicas the NameNode still lists; cross-cluster copy pointing a reader at the wrong cluster/block pool; test harnesses querying random block IDs.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/6637844d1536d194. Report an issue: GitHub.