{"record":{"id":"aa0ce9784ca55980","repo":"prestodb/presto","slug":"unable-to-read-binary-type-decimal-of-size-aa0ce9","errorCode":null,"errorMessage":"Unable to read BINARY type decimal of size ","messagePattern":"Unable to read BINARY type decimal of size ","errorType":"exception","errorClass":"ParquetDecodingException","httpStatus":null,"severity":"error","filePath":"presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/decoders/rle/ShortDecimalRLEDictionaryValuesDecoder.java","lineNumber":54,"sourceCode":"        requireNonNull(dictionary, \"dictionary is null\");\n        delegate = new BinaryRLEDictionaryValuesDecoder(bitWidth, inputStream, dictionary);\n    }\n\n    @Override\n    public void readNext(long[] values, int offset, int length)\n            throws IOException\n    {\n        ValueBuffer valueBuffer = delegate.readNext(length);\n        int bufferSize = valueBuffer.getBufferSize();\n        byte[] byteBuffer = new byte[bufferSize];\n        int[] offsets = new int[length + 1];\n        delegate.readIntoBuffer(byteBuffer, 0, offsets, 0, valueBuffer);\n\n        for (int i = 0; i < length; i++) {\n            int positionOffset = offsets[i];\n            int positionLength = offsets[i + 1] - positionOffset;\n            if (positionLength > 8) {\n                throw new ParquetDecodingException(\"Unable to read BINARY type decimal of size \" + positionLength + \" as a short decimal\");\n            }\n\n            values[offset + i] = getShortDecimalValue(byteBuffer, positionOffset, positionLength);\n        }\n    }\n\n    @Override\n    public void skip(int length)\n            throws IOException\n    {\n        delegate.skip(length);\n    }\n}\n","sourceCodeStart":36,"sourceCodeEnd":68,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-parquet/src/main/java/com/facebook/presto/parquet/batchreader/decoders/rle/ShortDecimalRLEDictionaryValuesDecoder.java#L36-L68","documentation":"ShortDecimalRLEDictionaryValuesDecoder decodes BINARY-encoded dictionary pages backing a short decimal (precision <= 18, must fit in a long / at most 8 bytes of two's-complement data). While copying variable-width dictionary values, it measures each entry's length from the offsets array; when an entry exceeds 8 bytes it throws ParquetDecodingException(\"Unable to read BINARY type decimal of size \" + positionLength + \" as a short decimal\"), because the value cannot be represented as a short decimal long. This indicates the file's declared decimal precision/scale does not match the actual stored byte widths.","triggerScenarios":"readNext() — called from nonNullCount, rleOnlyBlockReadHelper, hybridReadInBatchesHelper, tryReadingTooMany paths — encounters a dictionary BINARY entry whose byte length (offsets[i+1]-offsets[i]) is greater than 8, i.e., the physical bytes cannot fit into a short decimal.","commonSituations":"Hive/Impala/Spark writing a DECIMAL(>18) column that Presto's schema maps as a short decimal; schema drift where the table's precision was changed after data was written; corrupt variable-length dictionary entries; reading a BINARY column through a decimal-typed mapping by mistake.","solutions":["Fix the table/column schema so the Parquet column's decimal precision is <= 18 (or that Presto maps it as a long decimal), then re-run the query.","If the data is truly DECIMAL(19+), alter the Hive table column type so Presto reads it as a long decimal instead of a short decimal.","Validate with parquet-tools that the BINARY dictionary entries match the declared precision; re-export the file if data and schema disagree.","Cast/rewrite the column with a compatible precision on the writer side before Presto reads it."],"exampleFix":"// before: schema declares short decimal but data is wider\nCREATE TABLE t (d DECIMAL(10,2)) ... -- parquet actually holds DECIMAL(20,2)\n-- ParquetDecodingException: Unable to read BINARY type decimal of size 9 ...\n\n// after: align schema with physical data\nCREATE TABLE t (d DECIMAL(20,2)) ... -- read as long decimal","handlingStrategy":"validation","validationCode":"// Confirm the declared decimal precision fits a short decimal before reading\nint precision = column.getType().getPrecision();\nif (precision > 18) {\n    throw new IllegalArgumentException(\n        \"Column maps to short decimal but precision is \" + precision + \"; fix table schema\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    decoder.readNext(values, offset, length);\n} catch (ParquetDecodingException e) {\n    throw new PrestoException(PARQUET_CORRUPT_DATA,\n        \"BINARY decimal entry exceeds 8 bytes; check schema precision for \" + column, e);\n}","preventionTips":["Keep the Hive/Spark table's decimal precision in sync with the physical Parquet schema (precision <= 18 for short decimals).","Never change a column's decimal precision in the metastore after data is written without rewriting it.","Verify with parquet-tools that BINARY dictionary entry widths match the declared precision.","Map DECIMAL(19+) columns to long decimals, not BINARY-backed short decimals."],"tags":["parquet","decoding","decimal","schema-mismatch"],"backgroundTag":"parquet-decimal-precision-mismatch","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}