What causes "Decimal does not fit long (invalid table schema?)" in prestodb/presto?

Widen the table column to a precision that fits the data (e.g. DECIMAL(38,x), which reads via the 128-bit path) and rewrite the file, or fix the writer to emit only values within the declared precision.

prestodb/presto · error · OrcCorruptionException

Decimal does not fit long (invalid table schema?)

Error message

Decimal does not fit long (invalid table schema?)

What it means

Thrown by DecimalInputStream.nextLong when a varint-encoded short decimal needs more than 63 bits of payload (offset > 63, or offset==63 with a payload byte > 1), meaning the value cannot fit in a signed 64-bit long. The parenthetical hints the usual cause: the actual data does not match the declared column type (schema says SHORT_DECIMAL/DECIMAL(p<=18) but the file holds a larger value).

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/stream/DecimalInputStream.java:112

        }

        UnscaledDecimal128Arithmetic.pack(low, high, negative, result);
    }

    public long nextLong()
            throws IOException
    {
        long result = 0;
        int offset = 0;
        long b;
        do {
            b = input.read();
            if (b == -1) {
                throw new OrcCorruptionException(input.getOrcDataSourceId(), "Reading BigInteger past EOF");
            }
            long work = 0x7f & b;
            if (offset >= 63 && (offset != 63 || work > 1)) {
                throw new OrcCorruptionException(input.getOrcDataSourceId(), "Decimal does not fit long (invalid table schema?)");
            }
            result |= work << offset;
            offset += 7;
        }
        while (b >= 0x80);
        boolean isNegative = (result & 0x01) != 0;
        if (isNegative) {
            result += 1;
            result = -result;
            result = result >> 1;
            result |= 0x01L << 63;
        }
        else {
            result = result >> 1;
            result &= MAX_VALUE;
        }
        return result;
    }

View on GitHub (pinned to 55bb57d202)

Solutions

Widen the table column to a precision that fits the data (e.g. DECIMAL(38,x), which reads via the 128-bit path) and rewrite the file, or fix the writer to emit only values within the declared precision.
Identify offending rows by scanning source data for values outside the declared precision and clamp/cast them before writing.
Verify the ORC footer type with orc-dump to confirm writer/schema mismatch, then recreate the table with the correct ORC type.
If the bytes are actually corrupt, regenerate the file from source.

Example fix

-- before
CREATE TABLE t (amount DECIMAL(18,2), ...);
-- after: widen to hold values that exceed 18 digits
CREATE TABLE t (amount DECIMAL(38,2), ...);

Defensive patterns

Strategy: validation

Validate before calling

// caller-side guard before writing short decimals
if (value.precision() > 18) {
    value = value.setScale(scale, RoundingMode.HALF_UP);
    if (value.precision() > 18) {
        throw new IllegalArgumentException("does not fit DECIMAL(18,x): " + value);
    }
}

Try / catch

try {
    return readShortDecimal();
} catch (OrcCorruptionException e) {
    if (e.getMessage().contains("does not fit long")) {
        throw new SchemaMismatchException("column declared too narrow for data; widen to DECIMAL(38,x)", e);
    }
    throw e;
}

Prevention

Declare column precision no smaller than the widest value the source can produce.
Validate ranges in ETL against the table schema before writing.
When in doubt use 128-bit DECIMAL(38,x), which avoids the long path entirely.
Audit writer configs that drop or alter precision.

When it happens

Trigger: nextLong reads a varint whose continuation extends past byte offset 63 (or whose last allowed byte exceeds 1), i.e. the encoded magnitude exceeds Long range for a short-decimal column.

Common situations: Table schema declares DECIMAL(18,x) or similar but the writer stored bigger values; data imported from a system with unbounded precision (e.g. some ETL tools writing raw BigInteger); corrupt bytes extending a varint spuriously.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/c9506701f9db755f. Report an issue: GitHub.