prestodb/presto · error · IllegalArgumentException

maxOutputLength is incorrect, there is more data to be decom

Error message

maxOutputLength is incorrect, there is more data to be decompressed

What it means

InflateDecompressor (zlib) inflates a full ORC compression chunk into the output buffer in one call; if the Inflater does not report finished(), the maxOutputLength was too small for the compressed data, indicating the declared uncompressed size doesn't match the data — an IllegalArgumentException is thrown.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/zlib/InflateDecompressor.java:37

import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.util.zip.DataFormatException;
import java.util.zip.Inflater;

public class InflateDecompressor
        implements Decompressor
{
    @Override
    public int decompress(byte[] input, int inputOffset, int inputLength, byte[] output, int outputOffset, int maxOutputLength)
            throws MalformedInputException
    {
        Inflater inflater = new Inflater(true);
        inflater.setInput(input, inputOffset, inputLength);
        int uncompressedLength = 0;
        try {
            uncompressedLength = inflater.inflate(output, outputOffset, maxOutputLength);
            if (!inflater.finished()) {
                throw new IllegalArgumentException("maxOutputLength is incorrect, there is more data to be decompressed");
            }
        }
        catch (DataFormatException e) {
            throw new MalformedInputException(inputOffset, e.getMessage());
        }
        finally {
            inflater.end();
        }
        return uncompressedLength;
    }

    @Override
    public void decompress(ByteBuffer input, ByteBuffer output)
            throws MalformedInputException
    {
        if (input.isDirect() || output.isDirect() || !input.hasArray() || !output.hasArray()) {
            throw new IllegalArgumentException("Non-direct byte buffer backed by byte array required");
        }

View on GitHub (pinned to 55bb57d202)

Solutions

Verify the ORC file's compression metadata and integrity with orc-tools.
Re-read with correct chunk offsets (fresh metadata/split).
Check for corruption by re-transferring the file.
Ensure you use the same compression kind the file was written with.

Example fix

// before
byte[] out = new byte[declaredSize];
decompressor.decompress(in, 0, inLen, out, 0, declaredSize);
// after: size the output from the ORC chunk header (3-byte length)
int uncompressedLen = readChunkLength(headerBytes);
byte[] out = new byte[uncompressedLen];
decompressor.decompress(in, 0, inLen, out, 0, uncompressedLen);

Defensive patterns

Strategy: try-catch

Validate before calling

if (uncompressedSize <= 0 || uncompressedSize > MAX_CHUNK) {
    throw new IOException("implausible uncompressed chunk size: " + uncompressedSize);
}

Try / catch

try { compressor.decompress(in, off, len, out, 0, max); } catch (IllegalArgumentException e) {
    throw new DataCorruptionException("zlib block size mismatch", e);
}

Prevention

Size output buffers from the ORC 3-byte chunk header, not assumptions.
Match compression kind metadata with actual block content.
Validate blocks with orc-tools before serving queries.
Re-transfer corrupted files.

When it happens

Trigger: decompress invoked by the ORC stream reader with an output buffer smaller than the true inflated size — e.g., the chunk header length was wrong or the block was read with a wrong uncompressed-size assumption.

Common situations: Corrupted or misaligned compressed block; header length mis-parsed; non-zlib data fed to the zlib decompressor; file produced by a writer claiming different compression parameters.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/40f5f0c1df7cc7a0. Report an issue: GitHub.