{"record":{"id":"6f53191343349bc5","repo":"prestodb/presto","slug":"decoded-value-out-of-range-for-a-16bit-number","errorCode":null,"errorMessage":"Decoded value out of range for a 16bit number","messagePattern":"Decoded value out of range for a 16bit number","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV1.java","lineNumber":176,"sourceCode":"    @Override\n    public void next(short[] values, int items)\n            throws IOException\n    {\n        int offset = 0;\n        while (items > 0) {\n            if (used == numValuesInRun) {\n                numValuesInRun = 0;\n                used = 0;\n                readHeader();\n            }\n\n            int chunkSize = min(numValuesInRun - used, items);\n            if (repeat) {\n                for (int i = 0; i < chunkSize; i++) {\n                    long literal = repeatBase + ((used + i) * delta);\n                    short value = (short) literal;\n                    if (literal != value) {\n                        throw new OrcCorruptionException(input.getOrcDataSourceId(), \"Decoded value out of range for a 16bit number\");\n                    }\n                    values[offset + i] = value;\n                }\n            }\n            else {\n                for (int i = 0; i < chunkSize; i++) {\n                    long literal = input.readVarint(signed);\n                    short value = (short) literal;\n                    if (literal != value) {\n                        throw new OrcCorruptionException(input.getOrcDataSourceId(), \"Decoded value out of range for a 16bit number\");\n                    }\n                    values[offset + i] = value;\n                }\n            }\n            used += chunkSize;\n            offset += chunkSize;\n            items -= chunkSize;\n        }","sourceCodeStart":158,"sourceCodeEnd":194,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV1.java#L158-L194","documentation":"next(short[], items) decodes values as longs (run-mode reconstruction or varint literals) and narrows them to short (16-bit). When a decoded long does not fit in a short, the data contradicts the column's SMALLINT declaration. The library throws OrcCorruptionException instead of returning a value mangled by 16-bit truncation.","triggerScenarios":"next() called on a 16-bit column stream where repeatBase + (used + i) * delta (run mode) or a varint literal exceeds short range (-32768..32767) — corrupt bytes or the column actually holds INT/BIGINT-width data.","commonSituations":"SMALLINT columns whose writers encoded values beyond 16 bits, schema drift (column widened from SMALLINT to INT upstream), or corrupted ORC files after storage faults.","solutions":["Widen the column type to INT (or BIGINT) so decoding uses the wider path that can represent the values.","Validate the file with orc-tools; if bytes are corrupt, restore from backup or re-write the file.","Align the reader's schema with the writer's actual column type to prevent misinterpreting stream width.","Fix the producer if it emits out-of-range values for declared SMALLINT columns."],"exampleFix":"// before\nCREATE TABLE t (v SMALLINT) STORED AS ORC; // values exceed 16 bits\n// after\nALTER TABLE t CHANGE v v INT;","handlingStrategy":"validation","validationCode":"long minValue = sampleMinValue(orcFile, \"my_smallint_col\");\nlong maxValue = sampleMaxValue(orcFile, \"my_smallint_col\");\nif (minValue < Short.MIN_VALUE || maxValue > Short.MAX_VALUE) {\n    throw new SchemaMismatchException(\"values exceed SMALLINT range; use INT/BIGINT column\");\n}","typeGuard":"static boolean fitsShort(long v) { return v >= Short.MIN_VALUE && v <= Short.MAX_VALUE; }","tryCatchPattern":"try {\n    return shortStream.next(values, items);\n} catch (OrcCorruptionException e) {\n    if (e.getMessage().contains(\"out of range for a 16bit number\")) {\n        LOG.error(\"SMALLINT column holds values outside 16-bit range; re-read as INT\");\n        return intReader.next(); // re-decode via the 32-bit path\n    }\n    throw e;\n}","preventionTips":["Choose SMALLINT only when domain values are provably within -32768..32767.","Widen columns (SMALLINT → INT) proactively when producers may emit larger values.","Validate ORC files with orc-tools before registering them in the metastore.","Keep writer-side range checks in place so out-of-range values never reach encoded streams."],"tags":["orc","corruption","schema-mismatch","integer-overflow"],"backgroundTag":"value-out-of-range-for-type","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}