{"record":{"id":"7ee4bded490de28d","repo":"apache/cassandra","slug":"compression-failed","errorCode":null,"errorMessage":"Compression failed","messagePattern":"Compression failed","errorType":"exception","errorClass":"java.io.IOException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/io/compress/ZstdCompressorBase.java","lineNumber":162,"sourceCode":"        }\n    }\n\n    /**\n     * Compress using ByteBuffers\n     *\n     * @param input\n     * @param output\n     * @throws IOException\n     */\n    @Override\n    public void compress(ByteBuffer input, ByteBuffer output) throws IOException\n    {\n        try\n        {\n            Zstd.compress(output, input, compressionLevel(), ENABLE_CHECKSUM_FLAG);\n        } catch (Exception e)\n        {\n            throw new IOException(\"Compression failed\", e);\n        }\n    }\n\n    /**\n     * Check if the given compression level is valid. This can be a negative value as well.\n     *\n     * @param level compression level\n     */\n    public static void validateCompressionLevel(int level)\n    {\n        if (level < FAST_COMPRESSION_LEVEL || level > BEST_COMPRESSION_LEVEL)\n        {\n            throw new IllegalArgumentException(String.format(\"%s=%d is invalid\", COMPRESSION_LEVEL_OPTION_NAME, level));\n        }\n    }\n\n    /**\n     * Get the supplied compression level; otherwise, use the default","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/io/compress/ZstdCompressorBase.java#L144-L180","documentation":"The ByteBuffer-based ZstdCompressorBase.compress wraps any exception from Zstd.compress(output, input, level, checksum-flag) in an IOException 'Compression failed'. zstd-jni throws mainly when the destination buffer is too small for the compressed result (compression cannot shrink data enough), meaning the caller mis-sized the output buffer.","triggerScenarios":"Calling compress(ByteBuffer, ByteBuffer) where the output buffer's remaining capacity is less than zstd's worst-case requirement (roughly input size + overhead for incompressible data, plus frame/checksum overhead).","commonSituations":"Allocating the output buffer exactly equal to input length or to the expected compressed size, then feeding highly incompressible (random/already-compressed) data; insufficient buffer headroom for the checksum flag.","solutions":["Size the output buffer using Zstd.compressBound(inputLength) (or the compressor's initial/max compressed buffer length) rather than the input size.","Give extra headroom for the frame header and checksum (ENABLE_CHECKSUM_FLAG adds bytes).","Catch the IOException and inspect getCause() to confirm the failure is dstSize_tooSmall before changing buffer sizing."],"exampleFix":"// before\nByteBuffer out = ByteBuffer.allocate(input.remaining());\nzstdCompressor.compress(out, input);\n// after\nByteBuffer out = ByteBuffer.allocate((int) Zstd.compressBound(input.remaining()));\nzstdCompressor.compress(out, input);","handlingStrategy":"validation","validationCode":"int outCap = (int) com.github.luben.zstd.Zstd.compressBound(input.remaining());\nif (output.remaining() < outCap) throw new IllegalArgumentException(\"Output buffer too small: need \" + outCap);","typeGuard":null,"tryCatchPattern":"try {\n    compressor.compress(output, input);\n} catch (IOException e) {\n    if (e.getCause() != null && String.valueOf(e.getCause()).contains(\"dstSize\")) {\n        output = ByteBuffer.allocate((int) Zstd.compressBound(input.remaining()));\n    } else throw e;\n}","preventionTips":["Always size compression output buffers with Zstd.compressBound().","Add headroom for frame header and checksum bytes.","Test with incompressible data before deploying buffer sizing changes."],"tags":["zstd","compression","buffer-size","io"],"backgroundTag":"compression-failed","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}