apache/hadoop · error · PathIOException

Fail to get block MD5 for {}

Error message

Fail to get block MD5 for {}

What it means

ReplicatedFileChecksumComputer tries every located replica of a block; checksumBlock() returns false only when no replica could be read (all DataNodes dead, corrupt, or erroring), and the caller wraps that as a PathIOException naming the source file and the LocatedBlock. It means the file checksum could not be computed because one block is effectively unavailable, distinct from checksum-value mismatches.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/FileChecksumHelper.java:496

                                   ChecksumCombineMode combineMode)
        throws IOException {
      super(src, length, blockLocations, namenode, client, combineMode);
    }

    @Override
    void checksumBlocks() throws IOException {
      // get block checksum for each block
      for (blockIdx = 0;
           blockIdx < getLocatedBlocks().size() && getRemaining() >= 0;
           blockIdx++) {
        if (isRefetchBlocks()) {  // refetch to get fresh tokens
          refetchBlocks();
        }

        LocatedBlock locatedBlock = getLocatedBlocks().get(blockIdx);

        if (!checksumBlock(locatedBlock)) {
          throw new PathIOException(
              getSrc(), "Fail to get block MD5 for " + locatedBlock);
        }
      }
    }

    /**
     * Return true when sounds good to continue or retry, false when severe
     * condition or totally failed.
     */
    private boolean checksumBlock(LocatedBlock locatedBlock) {
      ExtendedBlock block = locatedBlock.getBlock();
      if (getRemaining() < block.getNumBytes()) {
        block.setNumBytes(getRemaining());
      }
      setRemaining(getRemaining() - block.getNumBytes());

      DatanodeInfo[] datanodes = locatedBlock.getLocations();

View on GitHub (pinned to 2add963021)

Solutions

1. Run hdfs fsck /path -files -blocks -locations to identify the unavailable block and its replicas
2. Restore/restart the DataNodes hosting the missing replicas, then retry the checksum
3. If replicas are truly lost, recover the file from source or accept data loss (fsck -move/-delete) — checksum verification is impossible without a readable replica
4. Retry after NameNode block reports refresh locations; transient decommissions self-heal

// Pre-check block health before checksum sweeps:
// hdfs fsck /path -files -blocks -locations  (expect 'HEALTHY')
// Programmatically: read the file fully first to confirm every block is servable.

int attempts = 0;
while (true) {
  try {
    return dfsClient.getFileChecksum(path);
  } catch (PathIOException e) {
    if (++attempts >= 3 || !String.valueOf(e.getMessage()).contains("Fail to get block MD5")) throw e;
    sleep(backoff(attempts)); // DataNodes may be restarting; locations refresh on retry
  }
}

When it happens

Trigger: getFileChecksum() on a replicated file where a block's every located replica fails: DataNodes down or decommissioning, replicas corrupt (checksum errors recorded during the attempt), block under reconstruction, or block tokens expired and refetch exhausted.

Common situations: Clusters with dead/decommissioned DataNodes that still own located replicas; hardware failures on all replicas' hosts; aggressive DataNode restarts during a checksum sweep; files with missing blocks reported by fsck.

Related errors

• Meta-data not found for {block}
• Checksum failed at {failedPos} for replica: {replica}
• Meta file for {} not found.
• Destination '{parentFile}' directory cannot be created
• Meta file not found, blockFile={blockFile}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4d7c7495b9f249a4. Report an issue: GitHub.