What causes "Incomplete file: header stipulated bytes but found only " in apache/cassandra?

Verify the file is complete and was fully written — re-copy or regenerate it from the source node/backup.

apache/cassandra · error · EOFException

Incomplete file: header stipulated <length> bytes but found

Error message

Incomplete file: header stipulated <length> bytes but found only <position>

What it means

CompressedFrameDataInputPlus reads length-prefixed, checksummed, Zstd-compressed frames from a channel. reBuffer() reads a frame header specifying the payload length; if the underlying channel hits EOF before that many bytes are read, this EOFException is thrown, meaning the file is truncated or corrupted.

Source

Thrown at src/java/org/apache/cassandra/io/util/CompressedFrameDataInputPlus.java:75

    protected void reBuffer() throws IOException
    {
        compressed.position(0);
        compressed.limit(SIZE_OF_HEADER);
        while (channel.read(compressed) >= 0 && compressed.hasRemaining());
        compressed.flip();
        long headerChecksum = compressed.getLong();
        int length = compressed.getShort();

        boolean decompress = length >= 0;
        if (!decompress)
            length = -1 - length;

        compressed.clear();
        compressed.limit(length);
        while (compressed.hasRemaining())
        {
            if (channel.read(compressed) < 0)
                throw new EOFException("Incomplete file: header stipulated " + length + " bytes but found only " + compressed.position());
        }
        compressed.flip();
        this.checksum.update(compressed);
        compressed.flip();
        long dataChecksum = checksum.getValue();
        if (headerChecksum != dataChecksum)
            throw new IOException("Invalid checksum: " + headerChecksum + " != " + dataChecksum);

        buffer.clear();
        if (decompress) compressor.uncompress(compressed, buffer);
        else buffer.put(compressed);
        buffer.flip();
    }

    public static <T> T read(File file, IVersionedSerializer<T> serializer) throws IOException
    {
        try (CompressedFrameDataInputPlus in = new CompressedFrameDataInputPlus(DEFAULT_FRAME_SIZE, file.newReadChannel(), ZstdCompressor.getOrCreate(DEFAULT_COMPRESSION_LEVEL), Crc.crc32()))
        {

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Verify the file is complete and was fully written — re-copy or regenerate it from the source node/backup.
Check file size on disk vs the size at write time; if truncated, restore from backup or rebuild the data.
Confirm the writer finished successfully (no crash/disk-full); re-run the write if the producer is under your control.
Catch EOFException and treat the file as unreadable, falling back to reconstruction rather than retrying.

Example fix

// before
MyMeta meta = CompressedFrameDataInputPlus.readOne(file, serializer);
// after
MyMeta meta;
try {
    meta = CompressedFrameDataInputPlus.readOne(file, serializer);
} catch (EOFException e) {
    logger.warn("Truncated file {}, rebuilding", file, e);
    meta = rebuildMeta();
}

Defensive patterns

Strategy: try-catch

Validate before calling

// before reading
if (!java.nio.file.Files.exists(file.toPath()) || java.nio.file.Files.size(file.toPath()) == 0)
    throw new IOException("File missing or empty: " + file);

Try / catch

try { return CompressedFrameDataInputPlus.readOne(file, serializer); }
catch (EOFException e) { logger.warn("Truncated file {}", file); return fallback(); }

Prevention

Only read these files after the writer has fully flushed and closed them
Copy SSTable component sets atomically (all files together)
Monitor for disk-full during writes
Verify file sizes against expected sizes after transfer

When it happens

Trigger: Calling CompressedFrameDataInputPlus.read/readOne/readList (or streaming from it) on a file that was truncated mid-frame — e.g. an incomplete flush during crash, a partial copy/rsync of the file, or reading a file still being written.

Common situations: Reading CompressedFrame files produced by CompressedFrameDataOutputPlus after an aborted write; copying SSTable-adjacent metadata files without completion markers; disk-full during write truncating the tail of the file; network filesystem serving stale file size metadata.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/cf2c1d184f167118. Report an issue: GitHub.