prestodb/presto · error · OrcCorruptionException

Read past end of RLE integer

Error message

Read past end of RLE integer

What it means

LongInputStreamV1 decodes ORC's Integer_RUN_LENGTH_V1 encoding. readHeader() reads one control byte that encodes run length and literal/run mode; if the underlying stream is already exhausted (input.read() == -1), the stream is structurally incomplete. The library throws OrcCorruptionException because a valid ORC column stream can never end mid-header.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV1.java:48

    private final boolean signed;
    private long repeatBase;
    private int numValuesInRun;
    private int delta;
    private int used;
    private boolean repeat;

    public LongInputStreamV1(OrcInputStream input, boolean signed)
    {
        this.input = input;
        this.signed = signed;
    }

    private void readHeader()
            throws IOException
    {
        int control = input.read();
        if (control == -1) {
            throw new OrcCorruptionException(input.getOrcDataSourceId(), "Read past end of RLE integer");
        }

        if (control < 0x80) {
            numValuesInRun = control + MIN_REPEAT_SIZE;
            used = 0;
            repeat = true;
            delta = input.read();
            if (delta == -1) {
                throw new OrcCorruptionException(input.getOrcDataSourceId(), "End of stream in RLE Integer");
            }

            // convert from 0 to 255 to -128 to 127 by converting to a signed byte
            delta = (byte) delta;
            repeatBase = input.readVarint(signed);
        }
        else {
            numValuesInRun = 0x100 - control;
            used = 0;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify file integrity (orc-tools `orccheck` or rewrite the file) — the ORC file is most likely truncated or corrupt.
2. Confirm the reader and writer ORC versions/protocols are compatible; re-write with a known-good writer if footer offsets disagree.
3. Check storage-side consistency (S3/HDFS byte ranges, split offsets/sizes) to ensure the reader receives the exact bytes the footer declares.
4. If files are being read while still written/uploaded, only read files after the writer commits/completes.

Example fix

// before: reading a file produced by an interrupted upload
orcFileReader.read(); // OrcCorruptionException: Read past end of RLE integer
// after: validate before reading
OrcWriteValidation validation = OrcWriteValidation.validate(file);
if (!validation.isValid()) {
    file = reFetchCompleteCopy(file); // obtain a fully-committed, non-truncated copy
}

long declaredSize = footerByteSize(orcFile);
long actualSize = orcFile.length();
if (actualSize < declaredSize) {
    throw new IllegalStateException("Truncated ORC file: " + actualSize + "/" + declaredSize + " bytes");
}

try (OrcDataSource src = FileOrcDataSource.open(path)) {
    OrcReader reader = new OrcReader(src, ...);
    return reader.read();
} catch (OrcCorruptionException e) {
    LOG.warn("ORC stream truncated, refetching complete copy", e);
    return readValidatedCopy(path); // re-fetch full file and validate before retry
}

When it happens

Trigger: Calling next() or skip() on a LongInputStreamV1 whose backing OrcDataSource has no bytes left when a new RLE header byte is needed — i.e. the declared stream length in the ORC footer is longer than the actual bytes present, or the reader advances past the stripe's stream boundary.

Common situations: Truncated ORC files (interrupted uploads/copies), reading a file written by a buggy or newer writer whose footer offsets disagree with actual data, wrong byte ranges fetched from split-aware storage (HDFS/S3) when offsets are miscomputed, or corrupt files after network glitches.

Related errors

• End of stream in RLE Integer
• Unexpected end of stream
• Validation failed
• Read past end of buffer RLE byte
• Reading RLE byte got EOF

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/17922295fe15d335. Report an issue: GitHub.