apache/cassandra · critical · java.io.IOException

Compressed lengths mismatch - %d bytes remain

Error message

Compressed lengths mismatch - %d bytes remain

What it means

The ByteBuffer-based LZ4Compressor.uncompress consumes the entire input buffer; when decompression finishes, any leftover bytes in the input indicate the input did not contain exactly one well-formed LZ4-compressed chunk. The compressor throws an IOException reporting how many bytes remain unconsumed.

Source

Thrown at src/java/org/apache/cassandra/io/compress/LZ4Compressor.java:190

                | ((input.get() & 0xFF) << 8)
                | ((input.get() & 0xFF) << 16)
                | ((input.get() & 0xFF) << 24);

        try
        {
            int compressedLength = input.remaining();
            decompressor.decompress(input, input.position(), input.remaining(), output, output.position(), decompressedLength);
            input.position(input.position() + compressedLength);
            output.position(output.position() + decompressedLength);
        }
        catch (LZ4Exception e)
        {
            throw new IOException(e);
        }

        if (input.remaining() > 0)
        {
            throw new IOException("Compressed lengths mismatch - "+input.remaining()+" bytes remain");
        }
    }

    public Set<String> supportedOptions()
    {
        return new HashSet<>(Arrays.asList(LZ4_HIGH_COMPRESSION_LEVEL, LZ4_COMPRESSOR_TYPE));
    }

    public static String validateCompressorType(String compressorType) throws ConfigurationException
    {
        if (compressorType == null)
            return DEFAULT_LZ4_COMPRESSOR_TYPE;

        if (!VALID_COMPRESSOR_TYPES.contains(compressorType))
        {
            throw new ConfigurationException(String.format("Invalid compressor type '%s' specified for LZ4 parameter '%s'. "
                                                           + "Valid options are %s.", compressorType, LZ4_COMPRESSOR_TYPE,
                                                           VALID_COMPRESSOR_TYPES.toString()));

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Verify the chunk data with `nodetool verify` and repair/scrub the affected SSTables or pull a clean copy from a replica.
2. Ensure the chunk length (compression chunk_length_kb) matches between writer and reader.
3. Check that the caller slices the input buffer to exactly one chunk before invoking uncompress.

Example fix

// before: passing whole file buffer
compressor.uncompress(fileBuffer, out);
// after: slice exactly one chunk
ByteBuffer chunk = fileBuffer.slice();
chunk.limit(compressedLen);
compressor.uncompress(chunk, out);

try {
    compressor.uncompress(chunkBuffer, output);
} catch (IOException e) {
    if (e.getMessage().startsWith("Compressed lengths mismatch")) {
        logger.warn("Chunk boundary/bytes corrupt: " + e.getMessage());
        // re-read from replica or fail the read
    } else throw e;
}

When it happens

Trigger: Calling LZ4Compressor.uncompress(ByteBuffer, ByteBuffer) with an input buffer that contains trailing garbage, concatenated frames, or a truncated/corrupt frame where LZ4 stops early and leaves input.remaining() > 0.

Common situations: Reading corrupt SSTable chunk data; passing a buffer with wrong chunk boundaries (incorrect chunk length configuration); data written by an incompatible producer.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• Decompressed lengths mismatch
• Decompression failed
• Decompression failed due to
• Not enough bytes to read size of %dth field %s
• Not enough bytes to read %dth field %s

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/86098d16175f7683. Report an issue: GitHub.