apache/cassandra · error · java.io.IOException

Compression failed

Error message

Compression failed

What it means

Thrown by ZstdDictionaryCompressor.compress when the Zstd JNI compression call (with or without a dictionary) throws an exception. It wraps the low-level failure into an IOException, aborting the chunk compression. Causes include a null/invalid dictionary, insufficient output buffer space, or a JNI-level error.

Source

Thrown at src/java/org/apache/cassandra/io/compress/ZstdDictionaryCompressor.java:210

    {
        if (dictionary == null)
        {
            super.compress(input, output);
            return;
        }

        try
        {
            // Zstd compressors expect only direct bytebuffer. See ZstdCompressorBase.preferredBufferType and supports
            int compressedSize = (int) Zstd.compressDirectByteBufferFastDict(output, output.position(), output.limit() - output.position(),
                                                                             input, input.position(), input.limit() - input.position(),
                                                                             dictionary.dictionaryForCompression(compressionLevel()));
            output.position(output.position() + compressedSize);
            input.position(input.limit());
        }
        catch (Exception e)
        {
            throw new IOException("Compression failed", e);
        }
    }

    @VisibleForTesting
    ZstdCompressionDictionary dictionary()
    {
        return dictionary;
    }

    @VisibleForTesting
    public static void invalidateCache()
    {
        instancePerDict.invalidateAll();
        instancePerDict.cleanUp();
    }
}

View on GitHub (pinned to 88fd0f6a0e)

Solutions

  1. Check the wrapped cause via getCause() for the underlying Zstd error
  2. Ensure the output buffer has at least Zstd.compressBound(inputLength) bytes remaining
  3. Verify the dictionary file exists, is a valid Zstd dictionary, and matches the configured compression parameters
  4. Retest with a lower compression level or without a dictionary to isolate the cause

Example fix

// before
ByteBuffer out = ByteBuffer.allocate(input.remaining());
compressor.compress(input, out);
// after
ByteBuffer out = ByteBuffer.allocate((int) Zstd.compressBound(input.remaining()));
compressor.compress(input, out);
Defensive patterns

Strategy: try-catch

Validate before calling

if (output.remaining() < Zstd.compressBound(input.remaining())) throw new IllegalArgumentException("compressed output buffer too small");

Type guard

null

Try / catch

try { compressor.compress(input, output); } catch (IOException e) { logger.error("zstd compress failed", e.getCause()); throw e; }

Prevention

When it happens

Trigger: Calling compress(ByteBuffer input, ByteBuffer output) (or the byte[] variant) when the output buffer has less remaining space than Zstd.compressBound(inputLength), or the loaded dictionary fails to initialize for compression (e.g. null dictionary with dictionary compression expected, or corrupted dictionary file).

Common situations: Sizing the compressed output buffer too small; dictionary training file corrupt or truncated; compression level out of accepted range; resource exhaustion in the Zstd context pool.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/da28e39452b34eae. Report an issue: GitHub.