prestodb/presto · error · OrcCorruptionException

Could not decompress all input (output buffer too small?)

Error message

Could not decompress all input (output buffer too small?)

What it means

Zlib decompression error in OrcZlibDecompressor.decompress: the inflater needs more output space than the configured max stream buffer size — the decompressed stripe/stream does not fit, so 'Could not decompress all input' fires. Usually a corrupt/truncated ORC stream or an implausibly high compression ratio.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcZlibDecompressor.java:58

        try {
            inflater.setInput(input, offset, length);
            byte[] buffer = output.initialize(Math.min(length * EXPECTED_COMPRESSION_RATIO, maxBufferSize));

            int uncompressedLength = 0;
            while (true) {
                uncompressedLength += inflater.inflate(buffer, uncompressedLength, buffer.length - uncompressedLength);
                if (inflater.finished() || buffer.length >= maxBufferSize) {
                    break;
                }
                int oldBufferSize = buffer.length;
                buffer = output.grow(Math.min(buffer.length * 2, maxBufferSize));
                if (buffer.length <= oldBufferSize) {
                    throw new IllegalStateException(String.format("Buffer failed to grow. Old size %d, current size %d", oldBufferSize, buffer.length));
                }
            }

            if (!inflater.finished()) {
                throw new OrcCorruptionException(orcDataSourceId, "Could not decompress all input (output buffer too small?)");
            }

            return uncompressedLength;
        }
        catch (DataFormatException e) {
            throw new OrcCorruptionException(e, orcDataSourceId, "Invalid compressed stream");
        }
        finally {
            inflater.end();
        }
    }

    @Override
    public String toString()
    {
        return "zlib";
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify the ORC file is not truncated or corrupted
2. Increase the ORC stream max buffer size config if the file legitimately decompresses larger
3. Reproduce with another ORC reader to confirm file integrity

When it happens

Trigger: Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcZlibDecompressor.java:58 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/96c411c4e4f5de23. Report an issue: GitHub.