apache/hadoop · critical · InvalidChecksumSizeException

The value %d does not map to a valid checksum Type

Error message

The value %d does not map to a valid checksum Type

What it means

The first checksum header byte is mapped through DataChecksum.Type.valueOf(int); only 0 (CHECKSUM_NULL), 1 (CHECKSUM_CRC32) and 2 (CHECKSUM_CRC32C) are legal ids. Any other value raises InvalidChecksumSizeException ('The value %d does not map to a valid checksum Type') — the type slot holds garbage, or a checksum type the running Hadoop version does not know.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java:185

   */
  public static DataChecksum newDataChecksum( DataInputStream in )
                                 throws IOException {
    int type = in.readByte();
    int bpc = in.readInt();
    DataChecksum summer = newDataChecksum(mapByteToChecksumType(type), bpc);
    if ( summer == null ) {
      throw new InvalidChecksumSizeException("Could not create DataChecksum "
          + "of type " + type + " with bytesPerChecksum " + bpc);
    }
    return summer;
  }

  private static Type mapByteToChecksumType(int type)
      throws InvalidChecksumSizeException{
    try {
      return Type.valueOf(type);
    } catch (IllegalArgumentException e) {
      throw new InvalidChecksumSizeException("The value "+type+" does not map"+
        " to a valid checksum Type");
    }
  }
  
  /**
   * Writes the checksum header to the output stream <i>out</i>.
   *
   * @param out output stream.
   * @throws IOException raised on errors performing I/O.
   */
  public void writeHeader( DataOutputStream out ) 
                           throws IOException { 
    out.writeByte( type.id );
    out.writeInt( bytesPerChecksum );
  }

  public byte[] getHeader() {
    byte[] header = new byte[getChecksumHeaderSize()];

View on GitHub (pinned to 2add963021)

Solutions

1. Confirm every reader/writer in the cluster understands the checksum type in use (ids 0, 1, 2 in this version)
2. Realign the stream: verify you are actually reading at a checksum header boundary
3. Run `hdfs fsck` and re-replicate corrupted blocks
4. Complete rolling upgrades before old components read data written with new checksum types

static boolean isValidChecksumTypeId(int id) {
  return id == DataChecksum.CHECKSUM_NULL
      || id == DataChecksum.CHECKSUM_CRC32
      || id == DataChecksum.CHECKSUM_CRC32C;
}
// before interpreting a header byte:
if (!isValidChecksumTypeId(typeByte & 0xff)) {
  throw new IOException("not positioned at a checksum header (type=" + (typeByte & 0xff) + ")");
}

static boolean isKnownChecksumType(int id) {
  try {
    org.apache.hadoop.util.DataChecksum.Type.valueOf(id);
    return true;
  } catch (IllegalArgumentException e) {
    return false;
  }
}

catch (InvalidChecksumSizeException e) {
  // type byte is not 0/1/2: either corruption or a newer checksum type than this build knows
  if (isVersionSkewSuspected()) logUpgradeAdvice();
  failOverToHealthyReplica();
}

When it happens

Trigger: Reading arbitrary/non-checksum bytes as a header because the stream or buffer is misaligned; corrupted block metadata filling the type byte with noise; a newer Hadoop writing a checksum type id that an older reader does not support (version skew).

Common situations: Rolling upgrades where datanodes and clients differ in version; manual edits or copying of block meta files; stream-positioning bugs in custom readers.

Related errors

• Could not create DataChecksum from the byte array of length
• Could not create DataChecksum from the byte array of length
• Expected to read {checksumSize} bytes from offset {offsetInC
• Expected to read {checksumSize} bytes from offset {offsetInC
• checksum verification failed: premature EOF

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/eb1e6615ccc94daa. Report an issue: GitHub.