prestodb/presto · error · ParquetDecodingException

Unable to read BINARY type decimal of size

Error message

Unable to read BINARY type decimal of size

What it means

BinaryShortDecimalDeltaValuesDecoder reads DELTA-encoded BINARY values and converts each to a short decimal (max precision 18, fitting in 8 bytes). If any value's binary length exceeds 8 bytes, it cannot represent a valid short decimal unscaled value, so a ParquetDecodingException is thrown listing the offending size.

Source

Thrown at presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/decoders/delta/BinaryShortDecimalDeltaValuesDecoder.java:57

        requireNonNull(bufferInputStream, "bufferInputStream is null");
        delegate = new BinaryDeltaValuesDecoder(encoding, valueCount, bufferInputStream);
    }

    @Override
    public void readNext(long[] values, int offset, int length)
            throws IOException
    {
        BinaryValuesDecoder.ValueBuffer valueBuffer = delegate.readNext(length);
        int bufferSize = valueBuffer.getBufferSize();
        byte[] byteBuffer = new byte[bufferSize];
        int[] offsets = new int[length + 1];
        delegate.readIntoBuffer(byteBuffer, 0, offsets, 0, valueBuffer);

        for (int i = 0; i < length; i++) {
            int positionOffset = offsets[i];
            int positionLength = offsets[i + 1] - positionOffset;
            if (positionLength > 8) {
                throw new ParquetDecodingException("Unable to read BINARY type decimal of size " + positionLength + " as a short decimal");
            }

            values[offset + i] = getShortDecimalValue(byteBuffer, positionOffset, positionLength);
        }
    }

    @Override
    public void skip(int length)
            throws IOException
    {
        checkArgument(length >= 0, "invalid length %s", length);
        delegate.skip(length);
    }

    @Override
    public long getRetainedSizeInBytes()
    {
        return INSTANCE_SIZE;

View on GitHub (pinned to 55bb57d202)

Solutions

  1. Verify the Parquet schema's DECIMAL precision/annotation matches the stored data (parquet-tools dump the column)
  2. Read the column as a long/decimal with wider precision if values exceed 18 digits, or use a session/catalog setting that maps to long decimals
  3. Re-write the file with the correct DECIMAL annotation matching the table schema
  4. If values should be short decimals, reject/repair rows with oversized values at write time

Example fix

// before: column annotated DECIMAL(38) but read as short decimal
// after: align read type with file precision
//   SELECT CAST(col AS DECIMAL(38,10)) FROM ... -- forces long-decimal decoder
// or rewrite file: SELECT CAST(col AS DECIMAL(18,6)) ...
Defensive patterns

Strategy: validation

Validate before calling

// Ensure the declared decimal precision fits a short decimal (<= 18):
// prec := file schema DECIMAL precision; if prec > 18 read as long decimal
// parquet-tools schema file.parquet | grep DECIMAL

Try / catch

try {
    block = decoder.readNext(length);
} catch (ParquetDecodingException e) {
    if (e.getMessage().startsWith("Unable to read BINARY type decimal of size")) {
        // re-run the read with a long-decimal decoder
    }
    throw e;
}

Prevention

When it happens

Trigger: readNext() iterates values and finds positionLength > 8 for a value, meaning the BINARY bytes encode an unscaled decimal too large for a short decimal column.

Common situations: Parquet file declares DECIMAL(precision<=18) but writer stored over-length binary values (schema drift, wrong annotation); reading a file whose decimal column was written as a longer precision decimal and the reader forces it into short decimal; hand-modified schemas where the file's precision (e.g. 38) no longer matches the table's.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/f616f5bd3575b805. Report an issue: GitHub.