prestodb/presto · error · OrcCorruptionException

Zstd requires buffer (%s) larger than max size (%s)

Error message

Zstd requires buffer (%s) larger than max size (%s)

What it means

Zstd decompression guard in OrcZstdDecompressor: the frame header's declared decompressed size exceeds the configured max buffer size, so the read is refused before decompressing. The %s pair gives required vs allowed size; indicates either a corrupt zstd stream or a max-buffer-size config that is too small.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcZstdDecompressor.java:50

    {
        this.orcDataSourceId = requireNonNull(orcDataSourceId, "orcDataSourceId is null");
        this.maxBufferSize = maxBufferSize;
        if (zstdJniDecompressionEnabled) {
            this.decompressor = new ZstdJniDecompressor();
        }
        else {
            this.decompressor = new ZstdDecompressor();
        }
    }

    @Override
    public int decompress(byte[] input, int offset, int length, OutputBuffer output)
            throws OrcCorruptionException
    {
        try {
            long uncompressedLength = ZstdDecompressor.getDecompressedSize(input, offset, length);
            if (uncompressedLength > maxBufferSize) {
                throw new OrcCorruptionException(orcDataSourceId, "Zstd requires buffer (%s) larger than max size (%s)", uncompressedLength, maxBufferSize);
            }

            byte[] buffer = output.initialize(toIntExact(uncompressedLength));
            return decompressor.decompress(input, offset, length, buffer, 0, buffer.length);
        }
        catch (MalformedInputException e) {
            throw new OrcCorruptionException(e, orcDataSourceId, "Invalid compressed stream");
        }
    }

    @Override
    public String toString()
    {
        return "zstd";
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

1. Increase the ORC stream max buffer size configuration
2. Verify the zstd stream is not corrupted (declared size sanity)
3. Recompress the data with smaller stripe/stream sizes

When it happens

Trigger: Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcZstdDecompressor.java:50 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/92e3de6e4c53b474. Report an issue: GitHub.