{"record":{"id":"40f5f0c1df7cc7a0","repo":"prestodb/presto","slug":"maxoutputlength-is-incorrect-there-is-more-data-t","errorCode":null,"errorMessage":"maxOutputLength is incorrect, there is more data to be decompressed","messagePattern":"maxOutputLength is incorrect, there is more data to be decompressed","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/zlib/InflateDecompressor.java","lineNumber":37,"sourceCode":"import java.nio.Buffer;\nimport java.nio.ByteBuffer;\nimport java.util.zip.DataFormatException;\nimport java.util.zip.Inflater;\n\npublic class InflateDecompressor\n        implements Decompressor\n{\n    @Override\n    public int decompress(byte[] input, int inputOffset, int inputLength, byte[] output, int outputOffset, int maxOutputLength)\n            throws MalformedInputException\n    {\n        Inflater inflater = new Inflater(true);\n        inflater.setInput(input, inputOffset, inputLength);\n        int uncompressedLength = 0;\n        try {\n            uncompressedLength = inflater.inflate(output, outputOffset, maxOutputLength);\n            if (!inflater.finished()) {\n                throw new IllegalArgumentException(\"maxOutputLength is incorrect, there is more data to be decompressed\");\n            }\n        }\n        catch (DataFormatException e) {\n            throw new MalformedInputException(inputOffset, e.getMessage());\n        }\n        finally {\n            inflater.end();\n        }\n        return uncompressedLength;\n    }\n\n    @Override\n    public void decompress(ByteBuffer input, ByteBuffer output)\n            throws MalformedInputException\n    {\n        if (input.isDirect() || output.isDirect() || !input.hasArray() || !output.hasArray()) {\n            throw new IllegalArgumentException(\"Non-direct byte buffer backed by byte array required\");\n        }","sourceCodeStart":19,"sourceCodeEnd":55,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/zlib/InflateDecompressor.java#L19-L55","documentation":"InflateDecompressor (zlib) inflates a full ORC compression chunk into the output buffer in one call; if the Inflater does not report finished(), the maxOutputLength was too small for the compressed data, indicating the declared uncompressed size doesn't match the data — an IllegalArgumentException is thrown.","triggerScenarios":"decompress invoked by the ORC stream reader with an output buffer smaller than the true inflated size — e.g., the chunk header length was wrong or the block was read with a wrong uncompressed-size assumption.","commonSituations":"Corrupted or misaligned compressed block; header length mis-parsed; non-zlib data fed to the zlib decompressor; file produced by a writer claiming different compression parameters.","solutions":["Verify the ORC file's compression metadata and integrity with orc-tools.","Re-read with correct chunk offsets (fresh metadata/split).","Check for corruption by re-transferring the file.","Ensure you use the same compression kind the file was written with."],"exampleFix":"// before\nbyte[] out = new byte[declaredSize];\ndecompressor.decompress(in, 0, inLen, out, 0, declaredSize);\n// after: size the output from the ORC chunk header (3-byte length)\nint uncompressedLen = readChunkLength(headerBytes);\nbyte[] out = new byte[uncompressedLen];\ndecompressor.decompress(in, 0, inLen, out, 0, uncompressedLen);","handlingStrategy":"try-catch","validationCode":"if (uncompressedSize <= 0 || uncompressedSize > MAX_CHUNK) {\n    throw new IOException(\"implausible uncompressed chunk size: \" + uncompressedSize);\n}","typeGuard":null,"tryCatchPattern":"try { compressor.decompress(in, off, len, out, 0, max); } catch (IllegalArgumentException e) {\n    throw new DataCorruptionException(\"zlib block size mismatch\", e);\n}","preventionTips":["Size output buffers from the ORC 3-byte chunk header, not assumptions.","Match compression kind metadata with actual block content.","Validate blocks with orc-tools before serving queries.","Re-transfer corrupted files."],"tags":["orc","zlib","compression","decompression"],"backgroundTag":"decompression-output-size-mismatch","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}