prestodb/presto · error · OrcCorruptionException

Invalid compressed stream

Error message

Invalid compressed stream

What it means

The Aircompressor Snappy decompressor throws MalformedInputException when the compressed bytes do not conform to the Snappy format. The decompressor translates that into OrcCorruptionException("Invalid compressed stream") with the cause chained.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/OrcSnappyDecompressor.java:51

    }

    @Override
    public int decompress(byte[] input, int offset, int length, OutputBuffer output)
            throws OrcCorruptionException
    {
        try {
            int uncompressedLength = SnappyDecompressor.getUncompressedLength(input, offset);
            if (uncompressedLength > maxBufferSize) {
                throw new OrcCorruptionException(orcDataSourceId, "Snappy requires buffer (%s) larger than max size (%s)", uncompressedLength, maxBufferSize);
            }

            // Snappy decompressor is more efficient if there's at least a long's worth of extra space
            // in the output buffer
            byte[] buffer = output.initialize(uncompressedLength + SIZE_OF_LONG);
            return decompressor.decompress(input, offset, length, buffer, 0, buffer.length);
        }
        catch (MalformedInputException e) {
            throw new OrcCorruptionException(e, orcDataSourceId, "Invalid compressed stream");
        }
    }

    @Override
    public String toString()
    {
        return "snappy";
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify the file checksum / re-transfer the file
2. Confirm the footer's compression kind matches the actual chunk encoding
3. Try reading with a different tool (e.g. orc-tools) to confirm corruption
4. Regenerate the file from the source if it is corrupt

// check file integrity before reading
FileStatus st = fs.getFileStatus(path);
if (st.getLen() != expectedLen || checksumMismatch(path)) { throw new IOException("corrupt file"); }

try { reader = new OrcReader(...); }
catch (OrcCorruptionException e) {
    if (e.getCause() instanceof MalformedInputException) {
        /* re-fetch file / regenerate before failing the job */
    }
}

When it happens

Trigger: decompress() calls decompressor.decompress and the underlying Snappy decoder detects malformed input at the given offset/length — corrupt chunk data or misidentified compression kind.

Common situations: Corrupted files (bit rot, bad transfers), reading a file whose compression kind metadata disagrees with the actual chunk encoding, writer bugs, mixing non-Snappy data flagged as Snappy.

Related errors

• Snappy requires buffer (%s) larger than max size (%s)
• HIVE_CURSOR_ERROR
• File has no columns
• Stripe encryption keys are missing, but file is encrypted
• Number of stripe encryption keys did not match number of enc

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/61f10247630ca281. Report an issue: GitHub.