prestodb/presto · error · OrcCorruptionException

Invalid RLEv2 encoded stream

Error message

Invalid RLEv2 encoded stream

What it means

In RLEv2 PATCHED_BASE decoding, the patch blob stores each patch entry using patchWidth + patchGapWidth bits; if their sum exceeds 64 bits the stream cannot be represented/decoded and Presto declares the RLEv2 stream invalid, throwing OrcCorruptionException. This check only fires when skipCorrupt is false.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV2.java:189

        // read the next base width number of bytes to extract base value
        long base = bytesToLongBE(input, baseWidth);
        long mask = (1L << ((baseWidth * 8) - 1));
        // if MSB of base value is 1 then base is negative value else positive
        if ((base & mask) != 0) {
            base = base & ~mask;
            base = -base;
        }

        // unpack the data blob
        long[] unpacked = new long[length];
        packer.unpack(unpacked, 0, length, fb, input);

        // unpack the patch blob
        long[] unpackedPatch = new long[patchListLength];

        if ((patchWidth + patchGapWidth) > 64 && !skipCorrupt) {
            throw new OrcCorruptionException(input.getOrcDataSourceId(), "Invalid RLEv2 encoded stream");
        }

        int bitSize = LongDecode.getClosestFixedBits(patchWidth + patchGapWidth);
        packer.unpack(unpackedPatch, 0, patchListLength, bitSize, input);

        // apply the patch directly when decoding the packed data
        int patchIndex = 0;
        long currentGap;
        long currentPatch;
        long patchMask = ((1L << patchWidth) - 1);
        currentGap = unpackedPatch[patchIndex] >>> patchWidth;
        currentPatch = unpackedPatch[patchIndex] & patchMask;
        long actualGap = 0;

        // special case: gap is >255 then patch value will be 0.
        // if gap is <=255 then patch value cannot be 0
        while (currentGap == 255 && currentPatch == 0) {
            actualGap += 255;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Treat the file/stripe as corrupt: validate with orc-tools and re-generate or re-copy the ORC file from a trusted source.
2. Identify and fix the writing tool: upgrade the producer (Hive/Spark/custom writer) to a conforming ORC writer version.
3. Re-run with ORC session properties that tolerate bad data (e.g. skip corrupt stripes / ignore corrupted blocks) if partial results are acceptable.
4. Check storage layer integrity (S3 checksums, HDFS block corruption reports) and repair affected blocks.
5. If the trigger is a framework bug, upgrade Presto to a version with updated RLEv2 handling.

Example fix

// before: corrupt file fails the whole query
SELECT * FROM t; -- Invalid RLEv2 encoded stream

// after: tolerate bad stripes (Presto session properties)
SET SESSION orc_broken_streams_on_file_boundary = true;
-- or re-write the file:
INSERT INTO t_fixed SELECT * FROM t_corrupt_source;

// Scan the file for decodable stripes before querying:
// java -jar orc-tools-*.jar scan data.orc  -- exits non-zero / logs on corrupt streams

try (OrcDataSource src = new FileOrcDataSource(path, config)) {
    new OrcReader(src, config).readAll();
} catch (OrcCorruptionException e) {
    // quarantine the file and re-ingest from source
}

When it happens

Trigger: readPatchedBaseValues (invoked from readValues -> next) encounters a header whose decoded patchWidth and patchGapWidth sum to > 64; this requires a corrupt/malformed header byte (thirdByte/fourthByte) since valid ORC writers never emit such widths.

Common situations: Bit-flipped or truncated bytes inside a stripe, files produced by broken third-party ORC writers, corrupted storage blocks, or reading a non-ORC/garbage file whose bytes accidentally decode into a patched-base header.

Related errors

• nanos field of an encoded timestamp in ORC must be between 0
• Read past end of RLE integer
• Decoded value out of range for a 32bit number
• Decoded value out of range for a 16bit number
• HIVE_INVALID_BUCKET_FILES

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/bbe79b96e8b613ba. Report an issue: GitHub.