prestodb/presto · error · OrcCorruptionException

Read past end of RLE integer

Error message

Read past end of RLE integer

What it means

LongInputStreamV2.readValues() starts decoding an RLE version 2 integer run by reading the header byte; if input.read() returns -1 the underlying ORC stream is exhausted, meaning the reader tried to consume more RLE data than the stream contains. Presto treats this as file corruption (OrcCorruptionException) because a well-formed stripe always contains the full run.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV2.java:66

    public LongInputStreamV2(OrcInputStream input, boolean signed, boolean skipCorrupt)
    {
        this.input = input;
        this.signed = signed;
        this.skipCorrupt = skipCorrupt;
        lastReadInputCheckpoint = input.getCheckpoint();
    }

    // This comes from the Apache Hive ORC code
    private void readValues()
            throws IOException
    {
        lastReadInputCheckpoint = input.getCheckpoint();

        // read the first 2 bits and determine the encoding type
        int firstByte = input.read();
        if (firstByte < 0) {
            throw new OrcCorruptionException(input.getOrcDataSourceId(), "Read past end of RLE integer");
        }

        int enc = (firstByte >>> 6) & 0x03;
        if (EncodingType.SHORT_REPEAT.ordinal() == enc) {
            readShortRepeatValues(firstByte);
        }
        else if (EncodingType.DIRECT.ordinal() == enc) {
            readDirectValues(firstByte);
        }
        else if (EncodingType.PATCHED_BASE.ordinal() == enc) {
            readPatchedBaseValues(firstByte);
        }
        else {
            readDeltaValues(firstByte);
        }
    }

    // This comes from the Apache Hive ORC code

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify file integrity: compare size/checksum against the source; re-download or re-copy the ORC file.
2. Confirm the file is complete (writer closed it); avoid reading files still being written.
3. Validate metadata with orc-tools; if stripe offsets are wrong, re-write the file with a current ORC writer.
4. Clear any stale cached blocks/metadata (Hive/Presto cache) and re-run the query.
5. Configure the connector to retry/skip corrupt stripes if occasional bad files are acceptable.

Example fix

// before: query fails reading a partially-uploaded object
SELECT count(*) FROM t; -- OrcCorruptionException: Read past end of RLE integer

// after: ensure complete copy, then query
aws s3 cp s3://src/data.orc s3://dst/data.orc --expected-size 123456789
SELECT count(*) FROM t;

// Before reading, check completeness:
long expected = sourceFileLength(); // authoritative size from manifest/checksum
long actual = Files.size(path);
if (actual < expected) throw new IllegalStateException("ORC file truncated: " + actual + " < " + expected);

try {
    orcReader.read(...);
} catch (PrestoException e) {
    if (e.getCause() instanceof OrcCorruptionException && e.getMessage().contains("Read past end of RLE integer")) {
        logger.warn("ORC stream truncated, skipping file/stripe");
        // skip or re-fetch the file, then retry
    } else {
        throw e;
    }
}

When it happens

Trigger: next(...) or skip(...) on a LongInputStreamV2 when the current stripe/chunk's data has run out: reading past the declared stream length due to a truncated file, wrong offsets/lengths in metadata, or a checkpoint seek into a bad position.

Common situations: ORC file truncated mid-stripe (partial upload/download), corrupted footer or stripe metadata, reading a file while it is still being written, cached/stale file blocks on a flaky storage layer (S3/HDFS), or a writer/reader version incompatibility that mis-states stream lengths.

Related errors

• nanos field of an encoded timestamp in ORC must be between 0
• Invalid RLEv2 encoded stream
• Decoded value out of range for a 32bit number
• Decoded value out of range for a 16bit number
• HIVE_INVALID_BUCKET_FILES

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/aa6409096b5e1dc5. Report an issue: GitHub.