prestodb/presto · error · OrcCorruptionException

Reading BigInteger past EOF

Error message

Reading BigInteger past EOF

What it means

Thrown by DecimalInputStream.nextLong when the varint-encoded BigInteger ends before the terminating byte (input.read() == -1). Short decimal values in ORC are varints; a stream that ends mid-varint cannot deliver the value, so the reader reports corruption rather than returning a wrong number.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/DecimalInputStream.java:108

            }
            else {
                low += 1;
            }
        }

        UnscaledDecimal128Arithmetic.pack(low, high, negative, result);
    }

    public long nextLong()
            throws IOException
    {
        long result = 0;
        int offset = 0;
        long b;
        do {
            b = input.read();
            if (b == -1) {
                throw new OrcCorruptionException(input.getOrcDataSourceId(), "Reading BigInteger past EOF");
            }
            long work = 0x7f & b;
            if (offset >= 63 && (offset != 63 || work > 1)) {
                throw new OrcCorruptionException(input.getOrcDataSourceId(), "Decimal does not fit long (invalid table schema?)");
            }
            result |= work << offset;
            offset += 7;
        }
        while (b >= 0x80);
        boolean isNegative = (result & 0x01) != 0;
        if (isNegative) {
            result += 1;
            result = -result;
            result = result >> 1;
            result |= 0x01L << 63;
        }
        else {
            result = result >> 1;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Validate with orc-tools and confirm the stream is truncated; re-copy the file from source with checksum verification.
2. Regenerate the ORC file from the underlying data.
3. Enable storage checksums so corruption is caught at transfer time.
4. Upgrade reader/writer versions to rule out spec mismatches producing wrong stream lengths.

// pre-check file completeness before opening
if (fileSize < footerReportedSize) { skipAndQuarantine(path); }

try {
    long v = decimalInputStream.nextLong();
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("Reading BigInteger past EOF")) {
        return Recover.copyFileAndRetry(path);
    }
    throw e;
}

When it happens

Trigger: nextLong (used by short-decimal reads) encounters EOF while still consuming varint bytes — the loop continues while b >= 0x80 and read() returns -1 first.

Common situations: Truncated ORC files (partial upload/copy); row group lengths inconsistent with actual stream bytes; corruption in unchecksummed storage; reading an in-progress ORC file.

Related errors

• Read past end of buffer RLE byte
• Reading RLE byte got EOF
• Unexpected end of stream
• EOF while reading unsigned vint
• Value is not null but decimal stream is not present

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/633dd0bab3f810be. Report an issue: GitHub.