apache/cassandra · error · java.io.IOException

Compression failed

Error message

Compression failed

What it means

The ByteBuffer-based ZstdCompressorBase.compress wraps any exception from Zstd.compress(output, input, level, checksum-flag) in an IOException 'Compression failed'. zstd-jni throws mainly when the destination buffer is too small for the compressed result (compression cannot shrink data enough), meaning the caller mis-sized the output buffer.

Source

Thrown at src/java/org/apache/cassandra/io/compress/ZstdCompressorBase.java:162

        }
    }

    /**
     * Compress using ByteBuffers
     *
     * @param input
     * @param output
     * @throws IOException
     */
    @Override
    public void compress(ByteBuffer input, ByteBuffer output) throws IOException
    {
        try
        {
            Zstd.compress(output, input, compressionLevel(), ENABLE_CHECKSUM_FLAG);
        } catch (Exception e)
        {
            throw new IOException("Compression failed", e);
        }
    }

    /**
     * Check if the given compression level is valid. This can be a negative value as well.
     *
     * @param level compression level
     */
    public static void validateCompressionLevel(int level)
    {
        if (level < FAST_COMPRESSION_LEVEL || level > BEST_COMPRESSION_LEVEL)
        {
            throw new IllegalArgumentException(String.format("%s=%d is invalid", COMPRESSION_LEVEL_OPTION_NAME, level));
        }
    }

    /**
     * Get the supplied compression level; otherwise, use the default

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Size the output buffer using Zstd.compressBound(inputLength) (or the compressor's initial/max compressed buffer length) rather than the input size.
2. Give extra headroom for the frame header and checksum (ENABLE_CHECKSUM_FLAG adds bytes).
3. Catch the IOException and inspect getCause() to confirm the failure is dstSize_tooSmall before changing buffer sizing.

Example fix

// before
ByteBuffer out = ByteBuffer.allocate(input.remaining());
zstdCompressor.compress(out, input);
// after
ByteBuffer out = ByteBuffer.allocate((int) Zstd.compressBound(input.remaining()));
zstdCompressor.compress(out, input);

int outCap = (int) com.github.luben.zstd.Zstd.compressBound(input.remaining());
if (output.remaining() < outCap) throw new IllegalArgumentException("Output buffer too small: need " + outCap);

try {
    compressor.compress(output, input);
} catch (IOException e) {
    if (e.getCause() != null && String.valueOf(e.getCause()).contains("dstSize")) {
        output = ByteBuffer.allocate((int) Zstd.compressBound(input.remaining()));
    } else throw e;
}

When it happens

Trigger: Calling compress(ByteBuffer, ByteBuffer) where the output buffer's remaining capacity is less than zstd's worst-case requirement (roughly input size + overhead for incompressible data, plus frame/checksum overhead).

Common situations: Allocating the output buffer exactly equal to input length or to the expected compressed size, then feeding highly incompressible (random/already-compressed) data; insufficient buffer headroom for the checksum flag.

Related errors

• Decompression failed due to
• Compression failed
• Compression exception
• Decompression failed
• Decompression failed due to

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/7ee4bded490de28d. Report an issue: GitHub.