prestodb/presto · error · OrcCorruptionException

End of stream in RLE Integer

Error message

End of stream in RLE Integer

What it means

In readHeader(), when the control byte indicates a repeating run (< 0x80), the stream must contain a follow-up signed delta byte (and a varint base). If the delta byte read returns -1, the stream ended exactly between the control byte and the run's delta — an impossible state for a well-formed ORC stream. The library throws OrcCorruptionException since the file's data is internally inconsistent or truncated.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV1.java:57

        this.input = input;
        this.signed = signed;
    }

    private void readHeader()
            throws IOException
    {
        int control = input.read();
        if (control == -1) {
            throw new OrcCorruptionException(input.getOrcDataSourceId(), "Read past end of RLE integer");
        }

        if (control < 0x80) {
            numValuesInRun = control + MIN_REPEAT_SIZE;
            used = 0;
            repeat = true;
            delta = input.read();
            if (delta == -1) {
                throw new OrcCorruptionException(input.getOrcDataSourceId(), "End of stream in RLE Integer");
            }

            // convert from 0 to 255 to -128 to 127 by converting to a signed byte
            delta = (byte) delta;
            repeatBase = input.readVarint(signed);
        }
        else {
            numValuesInRun = 0x100 - control;
            used = 0;
            repeat = false;
        }
    }

    @Override
    // This comes from the Apache Hive ORC code
    public long next()
            throws IOException
    {

View on GitHub (pinned to 55bb57d202)

Solutions

1. Re-fetch or restore the ORC file from a known-good source and validate it with orc-tools before reading.
2. Compare the stripe/footer stream lengths against actual readable bytes to locate where truncation occurred.
3. Ensure any copy/transfer of the file is byte-complete (check checksums such as MD5/CRC after transfer).
4. Upgrade/patch the ORC writer that produced the file if its stream encoding is malformed.

Example fix

// before
long len = file.length();
if (len < expectedFooterBytes) { /* proceeds anyway and fails mid-decode */ }
// after
if (fileSizeBytes < footerExpectedMinimumBytes) {
    throw new IllegalStateException("ORC file truncated: " + fileSizeBytes + " < " + footerExpectedMinimumBytes);
}

if (!OrcFileValidator.checkStripeStreams(orcFile)) { // stream byte lengths vs actual readable bytes
    orcFile = restoreKnownGoodCopy(orcFile);
}

try {
    return readColumnStream(stream);
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("End of stream in RLE Integer")) {
        throw new DataUnavailableException("ORC stream structurally truncated", e);
    }
    throw e;
}

When it happens

Trigger: next() or skip() reads a run-mode control byte as the very last byte of the stream; the delta byte fetch hits EOF. Happens when the column stream's byte length is one (or a few) short of what the run structure requires.

Common situations: Byte-level truncation of ORC files during copy/rsync, storage systems returning partial ranges, files written by non-conforming writers, or reading stale/corrupt blocks after a failed compaction.

Related errors

• Read past end of RLE integer
• Unexpected end of stream
• Validation failed
• Read past end of buffer RLE byte
• Reading RLE byte got EOF

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/d475aac69362dffe. Report an issue: GitHub.