apache/hadoop · error · CorruptMetaHeaderException

EOF while reading header from the metadata file. The meta fi

Error message

EOF while reading header from the metadata file. The meta file may be truncated or corrupt

What it means

BlockMetadataHeader.preadHeader reads the fixed-size meta header (version + checksum descriptor) from a block .meta file via FileChannel. A read() returning <= 0 while header bytes remain means the file is shorter than getHeaderSize() - truncated or empty. The CorruptMetaHeaderException marks the replica metadata unusable for serving or verification.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockMetadataHeader.java:124

    return header.getChecksum();
  }

  /**
   * Read the header without changing the position of the FileChannel.
   * This is used by the client for short-circuit reads.
   *
   * @param fc The FileChannel to read.
   * @return the Metadata Header.
   * @throws IOException on error.
   */
  public static BlockMetadataHeader preadHeader(FileChannel fc)
      throws IOException {
    final byte arr[] = new byte[getHeaderSize()];
    ByteBuffer buf = ByteBuffer.wrap(arr);

    while (buf.hasRemaining()) {
      if (fc.read(buf, buf.position()) <= 0) {
        throw new CorruptMetaHeaderException("EOF while reading header from "+
            "the metadata file. The meta file may be truncated or corrupt");
      }
    }
    short version = (short)((arr[0] << 8) | (arr[1] & 0xff));
    DataChecksum dataChecksum;
    try {
      dataChecksum = DataChecksum.newDataChecksum(arr, 2);
    } catch (InvalidChecksumSizeException e) {
      throw new CorruptMetaHeaderException("The block meta file header is "+
          "corrupt", e);
    }
    return new BlockMetadataHeader(version, dataChecksum);
  }

  /**
   * This reads all the fields till the beginning of checksum.
   * @return Metadata Header
   * @throws IOException

View on GitHub (pinned to 2add963021)

Solutions

1. Run 'hdfs fsck /' to locate corrupt/missing replicas and identify the affected block and files
2. Delete the affected block/meta pair (or its block subdirectory) so the NameNode schedules re-replication from other replicas
3. Fix the root cause: free disk space, stop the crash/hardware errors
4. If no other replica exists, restore from backup - the data is unrecoverable without a valid meta header

Path meta = blockMetaPath(blockId);
if (Files.size(meta) < BlockMetadataHeader.getHeaderSize()) {
  // truncated/empty header: quarantine before attempting preadHeader
  quarantine(meta);
}

catch (CorruptMetaHeaderException e) { deleteBlockAndMeta(block); requestReReplication(block); }

When it happens

Trigger: A DataNode or tooling path opens a .meta file whose length is below the header size: crash before the header was flushed, disk full during block creation, or a zero-length file left behind after kill -9.

Common situations: DN crash or reboot mid-write; full data disks; interrupted block moves/copies; bit rot on metadata volumes.

Related errors

• The block meta file header is corrupt
• Unknown op {} in data stream
• file VERSION is invalid.
• A disk IO error occurred
• file VERSION is invalid.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f9642d83ecc80abb. Report an issue: GitHub.