prestodb/presto · error · OrcCorruptionException

Unexpected end of stream

Error message

Unexpected end of stream

What it means

Thrown by ByteInputStream.next when, after readNextBlock, the internal buffer length is still 0 while the caller still needs more values — i.e. the stream is exhausted but the decoder expects `items` more bytes. It protects the reader from silently returning garbage when the byte stream ends prematurely.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/ByteInputStream.java:139

    public byte[] next(int items)
            throws IOException
    {
        byte[] values = new byte[items];
        next(values, items);
        return values;
    }

    public void next(byte[] values, int items)
            throws IOException
    {
        int outputOffset = 0;
        while (outputOffset < items) {
            if (offset == length) {
                readNextBlock();
            }
            if (length == 0) {
                throw new OrcCorruptionException(input.getOrcDataSourceId(), "Unexpected end of stream");
            }

            int chunkSize = min(items - outputOffset, length - offset);
            System.arraycopy(buffer, offset, values, outputOffset, chunkSize);

            outputOffset += chunkSize;
            offset += chunkSize;
        }
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

1. Run orc-tools meta to compare declared stream lengths with actual data; treat mismatch as proof of corruption.
2. Re-copy or regenerate the ORC file from the source data.
3. Check for mismatched reader/writer versions and upgrade Presto to a release that matches the file's ORC spec version.
4. Verify storage integrity (checksums, ETags) and re-enable them if disabled.
5. Narrow the query (e.g. read earlier row groups) to isolate which stripe is corrupt, then repair only that file.

// cheap pre-check: does the file end past the last stripe offset?
long footerEnd = f.length();
long lastStripeEnd = readLastStripeOffsetFromFooter(f);
if (lastStripeEnd >= footerEnd) { throw new IllegalStateException("stripes extend past EOF"); }

try {
    return reader.readBlock(column);
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("Unexpected end of stream")) {
        return fallback.readFromRebuiltCopy(path);
    }
    throw e;
}

When it happens

Trigger: Public next(values, items) loops copying bytes out of the RLE buffer; readNextBlock() sets offset=0 and on EOF leaves length==0, so the loop guard detects no data and throws before any caller (e.g. a Boolean/Byte stream reader) consumes more items than the stream holds.

Common situations: Row-group metadata declares more non-null values than the stream encodes; files truncated mid-strip; ORC files written by a buggy/older writer with wrong statistics; corruption during upload/storage.

Related errors

• Read past end of buffer RLE byte
• Reading RLE byte got EOF
• Reading BigInteger past EOF
• EOF while reading unsigned vint
• Read past end of RLE integer

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/344c7a5d4c513f99. Report an issue: GitHub.