{"record":{"id":"633dd0bab3f810be","repo":"prestodb/presto","slug":"reading-biginteger-past-eof","errorCode":null,"errorMessage":"Reading BigInteger past EOF","messagePattern":"Reading BigInteger past EOF","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/stream/DecimalInputStream.java","lineNumber":108,"sourceCode":"            }\n            else {\n                low += 1;\n            }\n        }\n\n        UnscaledDecimal128Arithmetic.pack(low, high, negative, result);\n    }\n\n    public long nextLong()\n            throws IOException\n    {\n        long result = 0;\n        int offset = 0;\n        long b;\n        do {\n            b = input.read();\n            if (b == -1) {\n                throw new OrcCorruptionException(input.getOrcDataSourceId(), \"Reading BigInteger past EOF\");\n            }\n            long work = 0x7f & b;\n            if (offset >= 63 && (offset != 63 || work > 1)) {\n                throw new OrcCorruptionException(input.getOrcDataSourceId(), \"Decimal does not fit long (invalid table schema?)\");\n            }\n            result |= work << offset;\n            offset += 7;\n        }\n        while (b >= 0x80);\n        boolean isNegative = (result & 0x01) != 0;\n        if (isNegative) {\n            result += 1;\n            result = -result;\n            result = result >> 1;\n            result |= 0x01L << 63;\n        }\n        else {\n            result = result >> 1;","sourceCodeStart":90,"sourceCodeEnd":126,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/stream/DecimalInputStream.java#L90-L126","documentation":"Thrown by DecimalInputStream.nextLong when the varint-encoded BigInteger ends before the terminating byte (input.read() == -1). Short decimal values in ORC are varints; a stream that ends mid-varint cannot deliver the value, so the reader reports corruption rather than returning a wrong number.","triggerScenarios":"nextLong (used by short-decimal reads) encounters EOF while still consuming varint bytes — the loop continues while b >= 0x80 and read() returns -1 first.","commonSituations":"Truncated ORC files (partial upload/copy); row group lengths inconsistent with actual stream bytes; corruption in unchecksummed storage; reading an in-progress ORC file.","solutions":["Validate with orc-tools and confirm the stream is truncated; re-copy the file from source with checksum verification.","Regenerate the ORC file from the underlying data.","Enable storage checksums so corruption is caught at transfer time.","Upgrade reader/writer versions to rule out spec mismatches producing wrong stream lengths."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// pre-check file completeness before opening\nif (fileSize < footerReportedSize) { skipAndQuarantine(path); }","typeGuard":null,"tryCatchPattern":"try {\n    long v = decimalInputStream.nextLong();\n} catch (OrcCorruptionException e) {\n    if (e.getMessage().contains(\"Reading BigInteger past EOF\")) {\n        return Recover.copyFileAndRetry(path);\n    }\n    throw e;\n}","preventionTips":["Verify checksums/ETags after transfer; reject incomplete copies.","Never expose files from unfinished writes (use staging + rename/commit).","Run orc-tools meta as an ingest gate.","Monitor storage for silent corruption with periodic audits."],"tags":["orc","decimal","eof","truncated-file"],"backgroundTag":"orc-file-corruption","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}