prestodb/presto · error · OrcCorruptionException

Decoded value out of range for a 16bit number

Error message

Decoded value out of range for a 16bit number

What it means

LongInputStreamV2.next(short[], items) narrows each buffered long RLEv2 literal to a short for SMALLINT columns; when the narrowing alters the value (literal outside -32768..32767) Presto throws OrcCorruptionException, since the stream contents contradict the 16-bit column type.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV2.java:401

    @Override
    public void next(short[] values, int items)
            throws IOException
    {
        int offset = 0;
        while (items > 0) {
            if (used == numLiterals) {
                numLiterals = 0;
                used = 0;
                readValues();
            }

            int chunkSize = min(numLiterals - used, items);
            for (int i = 0; i < chunkSize; i++) {
                long literal = literals[used + i];
                short value = (short) literal;
                if (literal != value) {
                    throw new OrcCorruptionException(input.getOrcDataSourceId(), "Decoded value out of range for a 16bit number");
                }
                values[offset + i] = value;
            }
            used += chunkSize;
            offset += chunkSize;
            items -= chunkSize;
        }
    }

    @Override
    public Class<LongStreamV2Checkpoint> getCheckpointType()
    {
        return LongStreamV2Checkpoint.class;
    }

    @Override
    public void seekToCheckpoint(LongStreamCheckpoint checkpoint)
            throws IOException

View on GitHub (pinned to 55bb57d202)

Solutions

1. Check the footer's column type and the actual value range; if values exceed smallint, map the column to INTEGER/BIGINT in Presto.
2. Validate the file with orc-tools and re-copy from source if bytes are corrupt.
3. Re-write the file with a conforming ORC writer if the producer emitted out-of-range shorts.
4. Use session/connector options to skip corrupt stripes when partial results are tolerable.
5. Upgrade the reader (Presto) or writer if a known ORC encoding bug applies.

Example fix

// before: SMALLINT schema over data with larger values
CREATE TABLE t (flag SMALLINT) WITH (external_location = 's3://bucket/bad.orc');

// after: read at natural width, narrow explicitly where safe
CREATE TABLE t (flag BIGINT) WITH (external_location = 's3://bucket/bad.orc');
SELECT try_cast(flag AS SMALLINT) FROM t;

// Confirm values fit smallint before reading as SMALLINT:
// orc-scan data.orc | awk '$1 < -32768 || $1 > 32767 {bad++} END {exit bad>0}'

try {
    session.execute("SELECT smallint_col FROM table");
} catch (PrestoException e) {
    if (e.getErrorCode().getCode() == OrcErrorCode.ORC_BAD_DATA.getCode()) {
        // fall back: read column as BIGINT, coerce with try_cast
    } else {
        throw e;
    }
}

When it happens

Trigger: Reading a SMALLINT ORC column where a decoded literal (from DIRECT, DELTA, PATCHED_BASE or SHORT_REPEAT runs) does not fit in 16 bits — typically corrupt data or a writer that emitted out-of-range values for a short column.

Common situations: Corrupted stripe bytes, non-conforming third-party ORC writers, schema drift (column actually contains int-range values but schema says smallint), or files damaged in transfer/storage.

Related errors

• nanos field of an encoded timestamp in ORC must be between 0
• Read past end of RLE integer
• Invalid RLEv2 encoded stream
• Decoded value out of range for a 32bit number
• HIVE_INVALID_BUCKET_FILES

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/6c8f06bae30f670e. Report an issue: GitHub.