apache/hadoop · critical · EOFException

Checksum file not a length multiple of checksum size in {} a

Error message

Checksum file not a length multiple of checksum size in {} at {} checksumpos: {} sumLenread: {}

What it means

While reading a file through ChecksumFs (FileContext's checksummed local fs), the code read N bytes from the sidecar .crc checksum file where N is not a multiple of 4 (the CRC entry size). A well-formed .crc is a header plus a whole number of 4-byte checksums, so this means the checksum file itself is truncated or malformed - corruption of the metadata, not the data.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFs.java:248

    protected int readChunk(long pos, byte[] buf, int offset, int len,
        byte[] checksum) throws IOException {
      boolean eof = false;
      if (needChecksum()) {
        assert checksum != null; // we have a checksum buffer
        assert checksum.length % CHECKSUM_SIZE == 0; // it is sane length
        assert len >= bytesPerSum; // we must read at least one chunk

        final int checksumsToRead = Math.min(
          len/bytesPerSum, // number of checksums based on len to read
          checksum.length / CHECKSUM_SIZE); // size of checksum buffer
        long checksumPos = getChecksumFilePos(pos); 
        if(checksumPos != sums.getPos()) {
          sums.seek(checksumPos);
        }

        int sumLenRead = sums.read(checksum, 0, CHECKSUM_SIZE * checksumsToRead);
        if (sumLenRead >= 0 && sumLenRead % CHECKSUM_SIZE != 0) {
          throw new EOFException("Checksum file not a length multiple of checksum size " +
                                 "in " + file + " at " + pos + " checksumpos: " + checksumPos +
                                 " sumLenread: " + sumLenRead );
        }
        if (sumLenRead <= 0) { // we're at the end of the file
          eof = true;
        } else {
          // Adjust amount of data to read based on how many checksum chunks we read
          len = Math.min(len, bytesPerSum * (sumLenRead / CHECKSUM_SIZE));
        }
      }
      if (pos != datas.getPos()) {
        datas.seek(pos);
      }
      int nread = readFully(datas, buf, offset, len);
      if (eof && nread > 0) {
        throw new ChecksumException("Checksum error: "+file+" at "+pos, pos);
      }
      return nread;

View on GitHub (pinned to 2add963021)

Solutions

Delete the stale sidecar checksum file (for a local path /p/f it is /p/.f.crc); readers then proceed without verification and the next create regenerates it.
Triage with checksums disabled: fc.setVerifyChecksum(false) then read the data - if the read succeeds the data is fine and only the .crc is bad.
Restore the file and its .crc together from your backup/replica.
If it recurs, check the volume with badblocks/smartctl - a repeatedly truncating .crc points at failing storage.

Example fix

// before
FSDataInputStream in = fc.open(path); // later read() throws EOFException:
// Checksum file not a length multiple of checksum size in /data/f at ... 

// after: drop the malformed sidecar, then read unverified
java.nio.file.Path crc = java.nio.file.Paths.get("/data/", ".f.crc");
java.nio.file.Files.deleteIfExists(crc);
FSDataInputStream in = fc.open(path);

Defensive patterns

Strategy: try-catch

Validate before calling

java.nio.file.Path crc = dir.resolve("." + fileName + ".crc");
if (java.nio.file.Files.exists(crc) && java.nio.file.Files.size(crc) % 4 != 0) {
  // malformed sidecar: delete or quarantine it before reading
}

Try / catch

try {
  in = fc.open(path);
  in.read(buf);
} catch (EOFException e) { // 'Checksum file not a length multiple...'
  // sidecar is corrupt: disable verification or delete the .crc and retry
}

Prevention

Always copy local Hadoop data with its hidden .crc files (rsync -a, cp -p), or delete all .crc files at once.
Never edit data files in place outside the checksummed fs API.
After any raw-fs append/truncate, remove the corresponding .crc.

When it happens

Trigger: Reading via FileContext/ChecksumFs when the .crc sidecar was truncated: the file pair was copied without the .crc being fully copied (cp of data file only partially), a disk-full event cut an earlier write short, or a non-Hadoop tool generated a bogus .crc. The exception fires in read() once the reader reaches the region whose checksums it must fetch.

Common situations: Data directories copied or rsync'd between nodes with hidden .crc files mishandled; interrupted write (kill -9 during create, full disk) leaving data file and .crc inconsistent; a previous raw-fs append/truncate (the unsupported-operation bypass) leaving a .crc whose length no longer matches the data.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e45946dcb8c15529. Report an issue: GitHub.