{"record":{"id":"d475aac69362dffe","repo":"prestodb/presto","slug":"end-of-stream-in-rle-integer","errorCode":null,"errorMessage":"End of stream in RLE Integer","messagePattern":"End of stream in RLE Integer","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV1.java","lineNumber":57,"sourceCode":"        this.input = input;\n        this.signed = signed;\n    }\n\n    private void readHeader()\n            throws IOException\n    {\n        int control = input.read();\n        if (control == -1) {\n            throw new OrcCorruptionException(input.getOrcDataSourceId(), \"Read past end of RLE integer\");\n        }\n\n        if (control < 0x80) {\n            numValuesInRun = control + MIN_REPEAT_SIZE;\n            used = 0;\n            repeat = true;\n            delta = input.read();\n            if (delta == -1) {\n                throw new OrcCorruptionException(input.getOrcDataSourceId(), \"End of stream in RLE Integer\");\n            }\n\n            // convert from 0 to 255 to -128 to 127 by converting to a signed byte\n            delta = (byte) delta;\n            repeatBase = input.readVarint(signed);\n        }\n        else {\n            numValuesInRun = 0x100 - control;\n            used = 0;\n            repeat = false;\n        }\n    }\n\n    @Override\n    // This comes from the Apache Hive ORC code\n    public long next()\n            throws IOException\n    {","sourceCodeStart":39,"sourceCodeEnd":75,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV1.java#L39-L75","documentation":"In readHeader(), when the control byte indicates a repeating run (< 0x80), the stream must contain a follow-up signed delta byte (and a varint base). If the delta byte read returns -1, the stream ended exactly between the control byte and the run's delta — an impossible state for a well-formed ORC stream. The library throws OrcCorruptionException since the file's data is internally inconsistent or truncated.","triggerScenarios":"next() or skip() reads a run-mode control byte as the very last byte of the stream; the delta byte fetch hits EOF. Happens when the column stream's byte length is one (or a few) short of what the run structure requires.","commonSituations":"Byte-level truncation of ORC files during copy/rsync, storage systems returning partial ranges, files written by non-conforming writers, or reading stale/corrupt blocks after a failed compaction.","solutions":["Re-fetch or restore the ORC file from a known-good source and validate it with orc-tools before reading.","Compare the stripe/footer stream lengths against actual readable bytes to locate where truncation occurred.","Ensure any copy/transfer of the file is byte-complete (check checksums such as MD5/CRC after transfer).","Upgrade/patch the ORC writer that produced the file if its stream encoding is malformed."],"exampleFix":"// before\nlong len = file.length();\nif (len < expectedFooterBytes) { /* proceeds anyway and fails mid-decode */ }\n// after\nif (fileSizeBytes < footerExpectedMinimumBytes) {\n    throw new IllegalStateException(\"ORC file truncated: \" + fileSizeBytes + \" < \" + footerExpectedMinimumBytes);\n}","handlingStrategy":"validation","validationCode":"if (!OrcFileValidator.checkStripeStreams(orcFile)) { // stream byte lengths vs actual readable bytes\n    orcFile = restoreKnownGoodCopy(orcFile);\n}","typeGuard":null,"tryCatchPattern":"try {\n    return readColumnStream(stream);\n} catch (OrcCorruptionException e) {\n    if (e.getMessage().contains(\"End of stream in RLE Integer\")) {\n        throw new DataUnavailableException(\"ORC stream structurally truncated\", e);\n    }\n    throw e;\n}","preventionTips":["Check file integrity (checksum/CRC) after transfers; a 1-byte truncation lands exactly in RLE headers.","Avoid reading files mid-write; gate reads on writer commit markers.","Validate stripe stream lengths against footer metadata before decoding.","Keep writer and reader ORC library versions consistent."],"tags":["orc","corruption","truncated-file","io"],"backgroundTag":"orc-corrupt-stream","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}