{"record":{"id":"86098d16175f7683","repo":"apache/cassandra","slug":"compressed-lengths-mismatch-d-bytes-remain","errorCode":null,"errorMessage":"Compressed lengths mismatch - %d bytes remain","messagePattern":"Compressed lengths mismatch - (.+?) bytes remain","errorType":"exception","errorClass":"java.io.IOException","httpStatus":null,"severity":"critical","filePath":"src/java/org/apache/cassandra/io/compress/LZ4Compressor.java","lineNumber":190,"sourceCode":"                | ((input.get() & 0xFF) << 8)\n                | ((input.get() & 0xFF) << 16)\n                | ((input.get() & 0xFF) << 24);\n\n        try\n        {\n            int compressedLength = input.remaining();\n            decompressor.decompress(input, input.position(), input.remaining(), output, output.position(), decompressedLength);\n            input.position(input.position() + compressedLength);\n            output.position(output.position() + decompressedLength);\n        }\n        catch (LZ4Exception e)\n        {\n            throw new IOException(e);\n        }\n\n        if (input.remaining() > 0)\n        {\n            throw new IOException(\"Compressed lengths mismatch - \"+input.remaining()+\" bytes remain\");\n        }\n    }\n\n    public Set<String> supportedOptions()\n    {\n        return new HashSet<>(Arrays.asList(LZ4_HIGH_COMPRESSION_LEVEL, LZ4_COMPRESSOR_TYPE));\n    }\n\n    public static String validateCompressorType(String compressorType) throws ConfigurationException\n    {\n        if (compressorType == null)\n            return DEFAULT_LZ4_COMPRESSOR_TYPE;\n\n        if (!VALID_COMPRESSOR_TYPES.contains(compressorType))\n        {\n            throw new ConfigurationException(String.format(\"Invalid compressor type '%s' specified for LZ4 parameter '%s'. \"\n                                                           + \"Valid options are %s.\", compressorType, LZ4_COMPRESSOR_TYPE,\n                                                           VALID_COMPRESSOR_TYPES.toString()));","sourceCodeStart":172,"sourceCodeEnd":208,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/io/compress/LZ4Compressor.java#L172-L208","documentation":"The ByteBuffer-based LZ4Compressor.uncompress consumes the entire input buffer; when decompression finishes, any leftover bytes in the input indicate the input did not contain exactly one well-formed LZ4-compressed chunk. The compressor throws an IOException reporting how many bytes remain unconsumed.","triggerScenarios":"Calling LZ4Compressor.uncompress(ByteBuffer, ByteBuffer) with an input buffer that contains trailing garbage, concatenated frames, or a truncated/corrupt frame where LZ4 stops early and leaves input.remaining() > 0.","commonSituations":"Reading corrupt SSTable chunk data; passing a buffer with wrong chunk boundaries (incorrect chunk length configuration); data written by an incompatible producer.","solutions":["Verify the chunk data with `nodetool verify` and repair/scrub the affected SSTables or pull a clean copy from a replica.","Ensure the chunk length (compression chunk_length_kb) matches between writer and reader.","Check that the caller slices the input buffer to exactly one chunk before invoking uncompress."],"exampleFix":"// before: passing whole file buffer\ncompressor.uncompress(fileBuffer, out);\n// after: slice exactly one chunk\nByteBuffer chunk = fileBuffer.slice();\nchunk.limit(compressedLen);\ncompressor.uncompress(chunk, out);","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    compressor.uncompress(chunkBuffer, output);\n} catch (IOException e) {\n    if (e.getMessage().startsWith(\"Compressed lengths mismatch\")) {\n        logger.warn(\"Chunk boundary/bytes corrupt: \" + e.getMessage());\n        // re-read from replica or fail the read\n    } else throw e;\n}","preventionTips":["Slice the input buffer to exactly one chunk before calling uncompress.","Keep chunk_length_kb consistent between writes and reads.","Detect corruption early with `nodetool verify`/`scrub`."],"tags":["lz4","decompression","data-corruption","bytebuffer"],"backgroundTag":"checksum-mismatch","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}