apache/hadoop · error · IOException

Partial CRC {} does not match value computed the last time

Error message

Partial CRC {} does not match value computed the  last time file was closed {}

What it means

Error "Partial CRC {} does not match value computed the last time file was closed {}" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java:1232

      instr.readChecksumFully(crcbuf, 0, crcbuf.length);
    }

    // compute crc of partial chunk from data read in the block file.
    final Checksum partialCrc = DataChecksum.newDataChecksum(
        diskChecksum.getChecksumType(), diskChecksum.getBytesPerChecksum());
    partialCrc.update(buf, 0, sizePartialChunk);
    if (LOG.isDebugEnabled()) {
      LOG.debug("Read in partial CRC chunk from disk for " + block);
    }

    // paranoia! verify that the pre-computed crc matches what we
    // recalculated just now
    if (partialCrc.getValue() != checksum2long(crcbuf)) {
      String msg = "Partial CRC " + partialCrc.getValue() +
                   " does not match value computed the " +
                   " last time file was closed " +
                   checksum2long(crcbuf);
      throw new IOException(msg);
    }
    return partialCrc;
  }

  /** The caller claims the ownership of the replica handler. */
  private ReplicaHandler claimReplicaHandler() {
    ReplicaHandler handler = replicaHandler;
    replicaHandler = null;
    return handler;
  }

  private enum PacketResponderType {
    NON_PIPELINE, LAST_IN_PIPELINE, HAS_DOWNSTREAM_IN_PIPELINE
  }

  /**
   * Processes responses from downstream datanodes in the pipeline
   * and sends back replies to the originator.

View on GitHub (pinned to 2add963021)

Solutions

1. Check disk and memory health on the DataNode; a partial CRC mismatch on close indicates data written earlier no longer matches its recorded CRC.
2. Delete the corrupt replica so the NameNode schedules re-replication from a healthy copy.

When it happens

Trigger: While finalizing a partially written block, the recomputed CRC of the last partial chunk differs from the CRC computed when the file was last closed.

Common situations: The CRC of a partially written chunk differed from the CRC recorded when the file was last closed, indicating on-disk corruption. Check the storage volume health.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8f4f96a1684141f4. Report an issue: GitHub.