apache/hadoop · error · CorruptMetaHeaderException

The block meta file header is corrupt

Error message

The block meta file header is corrupt

What it means

preadHeader successfully read the raw header bytes from the FileChannel, but DataChecksum.newDataChecksum(arr, 2) threw InvalidChecksumSizeException: the checksum descriptor at offset 2 encodes an invalid checksum size. Unlike the EOF case this is corruption of the header bytes, not truncation; it is wrapped in CorruptMetaHeaderException.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockMetadataHeader.java:133

   * @throws IOException on error.
   */
  public static BlockMetadataHeader preadHeader(FileChannel fc)
      throws IOException {
    final byte arr[] = new byte[getHeaderSize()];
    ByteBuffer buf = ByteBuffer.wrap(arr);

    while (buf.hasRemaining()) {
      if (fc.read(buf, buf.position()) <= 0) {
        throw new CorruptMetaHeaderException("EOF while reading header from "+
            "the metadata file. The meta file may be truncated or corrupt");
      }
    }
    short version = (short)((arr[0] << 8) | (arr[1] & 0xff));
    DataChecksum dataChecksum;
    try {
      dataChecksum = DataChecksum.newDataChecksum(arr, 2);
    } catch (InvalidChecksumSizeException e) {
      throw new CorruptMetaHeaderException("The block meta file header is "+
          "corrupt", e);
    }
    return new BlockMetadataHeader(version, dataChecksum);
  }

  /**
   * This reads all the fields till the beginning of checksum.
   * @return Metadata Header
   * @throws IOException
   */
  public static BlockMetadataHeader readHeader(DataInputStream in)
      throws IOException {
    try {
      return readHeader(in.readShort(), in);
    } catch (EOFException eof) {
      // The attempt to read the header threw EOF, indicating there are not
      // enough bytes in the meta file for the header.
      throw new CorruptMetaHeaderException("EOF while reading header from meta"+

View on GitHub (pinned to 2add963021)

Solutions

1. Identify the affected blocks with 'hdfs fsck /'
2. Delete the corrupt block/meta pair so HDFS re-replicates from healthy replicas
3. Investigate hardware/filesystem errors on the volume that produced the corrupt header

static boolean isCorruptMetaHeader(Throwable t) {
  return t instanceof CorruptMetaHeaderException;
}

try {
  BlockMetadataHeader header = BlockMetadataHeader.preadHeader(fc);
} catch (CorruptMetaHeaderException e) {
  // header bytes are garbage: drop the replica pair and let NN re-replicate
  deleteBlockAndMeta(block);
}

When it happens

Trigger: Same FileChannel read path, with a .meta file long enough but whose checksum type/length bytes are garbage: wrong checksum type byte, insane header length, or unrelated data overwriting the file.

Common situations: Disk corruption; a wrong file sitting at the meta path; partially overwritten .meta files after crashes or buggy tooling.

Related errors

• Expected to read {checksumSize} bytes from offset {offsetInC
• Expected to read {checksumSize} bytes from offset {offsetInC
• checksum verification failed: premature EOF
• Checksum verification failed for the block ${blockFileName}:
• fetchBlockByteRange(). Got a checksum exception for {} at {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0b765a6ca25543f5. Report an issue: GitHub.