{"record":{"id":"17922295fe15d335","repo":"prestodb/presto","slug":"read-past-end-of-rle-integer","errorCode":null,"errorMessage":"Read past end of RLE integer","messagePattern":"Read past end of RLE integer","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV1.java","lineNumber":48,"sourceCode":"    private final boolean signed;\n    private long repeatBase;\n    private int numValuesInRun;\n    private int delta;\n    private int used;\n    private boolean repeat;\n\n    public LongInputStreamV1(OrcInputStream input, boolean signed)\n    {\n        this.input = input;\n        this.signed = signed;\n    }\n\n    private void readHeader()\n            throws IOException\n    {\n        int control = input.read();\n        if (control == -1) {\n            throw new OrcCorruptionException(input.getOrcDataSourceId(), \"Read past end of RLE integer\");\n        }\n\n        if (control < 0x80) {\n            numValuesInRun = control + MIN_REPEAT_SIZE;\n            used = 0;\n            repeat = true;\n            delta = input.read();\n            if (delta == -1) {\n                throw new OrcCorruptionException(input.getOrcDataSourceId(), \"End of stream in RLE Integer\");\n            }\n\n            // convert from 0 to 255 to -128 to 127 by converting to a signed byte\n            delta = (byte) delta;\n            repeatBase = input.readVarint(signed);\n        }\n        else {\n            numValuesInRun = 0x100 - control;\n            used = 0;","sourceCodeStart":30,"sourceCodeEnd":66,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV1.java#L30-L66","documentation":"LongInputStreamV1 decodes ORC's Integer_RUN_LENGTH_V1 encoding. readHeader() reads one control byte that encodes run length and literal/run mode; if the underlying stream is already exhausted (input.read() == -1), the stream is structurally incomplete. The library throws OrcCorruptionException because a valid ORC column stream can never end mid-header.","triggerScenarios":"Calling next() or skip() on a LongInputStreamV1 whose backing OrcDataSource has no bytes left when a new RLE header byte is needed — i.e. the declared stream length in the ORC footer is longer than the actual bytes present, or the reader advances past the stripe's stream boundary.","commonSituations":"Truncated ORC files (interrupted uploads/copies), reading a file written by a buggy or newer writer whose footer offsets disagree with actual data, wrong byte ranges fetched from split-aware storage (HDFS/S3) when offsets are miscomputed, or corrupt files after network glitches.","solutions":["Verify file integrity (orc-tools `orccheck` or rewrite the file) — the ORC file is most likely truncated or corrupt.","Confirm the reader and writer ORC versions/protocols are compatible; re-write with a known-good writer if footer offsets disagree.","Check storage-side consistency (S3/HDFS byte ranges, split offsets/sizes) to ensure the reader receives the exact bytes the footer declares.","If files are being read while still written/uploaded, only read files after the writer commits/completes."],"exampleFix":"// before: reading a file produced by an interrupted upload\norcFileReader.read(); // OrcCorruptionException: Read past end of RLE integer\n// after: validate before reading\nOrcWriteValidation validation = OrcWriteValidation.validate(file);\nif (!validation.isValid()) {\n    file = reFetchCompleteCopy(file); // obtain a fully-committed, non-truncated copy\n}","handlingStrategy":"try-catch","validationCode":"long declaredSize = footerByteSize(orcFile);\nlong actualSize = orcFile.length();\nif (actualSize < declaredSize) {\n    throw new IllegalStateException(\"Truncated ORC file: \" + actualSize + \"/\" + declaredSize + \" bytes\");\n}","typeGuard":null,"tryCatchPattern":"try (OrcDataSource src = FileOrcDataSource.open(path)) {\n    OrcReader reader = new OrcReader(src, ...);\n    return reader.read();\n} catch (OrcCorruptionException e) {\n    LOG.warn(\"ORC stream truncated, refetching complete copy\", e);\n    return readValidatedCopy(path); // re-fetch full file and validate before retry\n}","preventionTips":["Only read ORC files after the writer fully commits (no partial uploads).","Verify file checksums after any copy/transfer before querying.","Periodically validate files with orc-tools orccheck.","Ensure split offsets/sizes from storage (S3/HDFS) match the footer's declared stream extents."],"tags":["orc","corruption","io","truncated-file"],"backgroundTag":"orc-corrupt-stream","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}