prestodb/presto · error · RuntimeException

Zstd JNI decompressor failed with

Error message

Zstd JNI decompressor failed with 

What it means

Zstd native decompression is invoked through JNI via Zstd.decompressByteArray; when the native library reports an error code, this is translated into a RuntimeException containing the Zstd error name (e.g. DESTINATION_BUFFER_TOO_SMALL, CORRUPTION_DETECTED).

Source

Thrown at presto-spark-base/src/main/java/com/facebook/presto/spark/util/PrestoSparkUtils.java:196

                int written = compress(input.array(), inputOffset, input.remaining(), output.array(), outputOffset, output.remaining());
                ((Buffer) output).position(output.position() + written);
            }
        };
    }

    private static PageDecompressor createPageDecompressor()
    {
        // based on ZstdJniDecompressor
        return new PageDecompressor()
        {
            @Override
            public int decompress(byte[] input, int inputOffset, int inputLength, byte[] output, int outputOffset, int maxOutputLength)
            {
                long size = Zstd.decompressByteArray(output, 0, maxOutputLength, input, inputOffset, inputLength);
                if (Zstd.isError(size)) {
                    String errorName = Zstd.getErrorName(size);
                    throw new RuntimeException("Zstd JNI decompressor failed with " + errorName);
                }
                return toIntExact(size);
            }

            @Override
            public void decompress(ByteBuffer input, ByteBuffer output)
            {
                if (input.isDirect() || output.isDirect() || !input.hasArray() || !output.hasArray()) {
                    throw new IllegalArgumentException("Non-direct byte buffer backed by byte array required");
                }
                int inputOffset = input.arrayOffset() + input.position();
                int outputOffset = output.arrayOffset() + output.position();

                int written = decompress(input.array(), inputOffset, input.remaining(), output.array(), outputOffset, output.remaining());
                ((Buffer) output).position(output.position() + written);
            }
        };
    }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify the input bytes are intact Zstd frames (check for truncation/corruption)
2. Ensure the output buffer is at least the original uncompressed size
3. Confirm compression and decompression sides use the same codec and Zstd version
4. Regenerate the corrupted spill/shuffle data

// verify input integrity before decompress
if (inputLength == 0 || output == null || maxOutputLength <= 0) throw new IllegalArgumentException("bad zstd input");

try { int written = decompress(input, inOff, inLen, output, outOff, maxLen); } catch (RuntimeException e) { if (e.getMessage().startsWith("Zstd JNI decompressor failed")) { /* treat data as corrupted: re-fetch or regenerate */ } else throw e; }

When it happens

Trigger: Calling PrestoSparkUtils decompress(byte[],...) with corrupt, truncated, or non-Zstd input data, or an output buffer smaller than the decompressed size.

Common situations: Shuffle/spill data corrupted on disk or truncated writes; decompressing data produced by a different codec or version; under-sized output buffer allocated from a stale/incorrect size header.

Related errors

• Invalid compressed stream
• Zstd JNI decompressor failed with
• Non-direct byte buffer backed by byte array required
• Invalid position %s and length %s in block with %s positions
• Invalid positions array size %d, actual position count is %d

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/f6b9b2279c65b699. Report an issue: GitHub.