{"record":{"id":"aa6409096b5e1dc5","repo":"prestodb/presto","slug":"read-past-end-of-rle-integer-aa6409","errorCode":null,"errorMessage":"Read past end of RLE integer","messagePattern":"Read past end of RLE integer","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV2.java","lineNumber":66,"sourceCode":"\n    public LongInputStreamV2(OrcInputStream input, boolean signed, boolean skipCorrupt)\n    {\n        this.input = input;\n        this.signed = signed;\n        this.skipCorrupt = skipCorrupt;\n        lastReadInputCheckpoint = input.getCheckpoint();\n    }\n\n    // This comes from the Apache Hive ORC code\n    private void readValues()\n            throws IOException\n    {\n        lastReadInputCheckpoint = input.getCheckpoint();\n\n        // read the first 2 bits and determine the encoding type\n        int firstByte = input.read();\n        if (firstByte < 0) {\n            throw new OrcCorruptionException(input.getOrcDataSourceId(), \"Read past end of RLE integer\");\n        }\n\n        int enc = (firstByte >>> 6) & 0x03;\n        if (EncodingType.SHORT_REPEAT.ordinal() == enc) {\n            readShortRepeatValues(firstByte);\n        }\n        else if (EncodingType.DIRECT.ordinal() == enc) {\n            readDirectValues(firstByte);\n        }\n        else if (EncodingType.PATCHED_BASE.ordinal() == enc) {\n            readPatchedBaseValues(firstByte);\n        }\n        else {\n            readDeltaValues(firstByte);\n        }\n    }\n\n    // This comes from the Apache Hive ORC code","sourceCodeStart":48,"sourceCodeEnd":84,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/stream/LongInputStreamV2.java#L48-L84","documentation":"LongInputStreamV2.readValues() starts decoding an RLE version 2 integer run by reading the header byte; if input.read() returns -1 the underlying ORC stream is exhausted, meaning the reader tried to consume more RLE data than the stream contains. Presto treats this as file corruption (OrcCorruptionException) because a well-formed stripe always contains the full run.","triggerScenarios":"next(...) or skip(...) on a LongInputStreamV2 when the current stripe/chunk's data has run out: reading past the declared stream length due to a truncated file, wrong offsets/lengths in metadata, or a checkpoint seek into a bad position.","commonSituations":"ORC file truncated mid-stripe (partial upload/download), corrupted footer or stripe metadata, reading a file while it is still being written, cached/stale file blocks on a flaky storage layer (S3/HDFS), or a writer/reader version incompatibility that mis-states stream lengths.","solutions":["Verify file integrity: compare size/checksum against the source; re-download or re-copy the ORC file.","Confirm the file is complete (writer closed it); avoid reading files still being written.","Validate metadata with orc-tools; if stripe offsets are wrong, re-write the file with a current ORC writer.","Clear any stale cached blocks/metadata (Hive/Presto cache) and re-run the query.","Configure the connector to retry/skip corrupt stripes if occasional bad files are acceptable."],"exampleFix":"// before: query fails reading a partially-uploaded object\nSELECT count(*) FROM t; -- OrcCorruptionException: Read past end of RLE integer\n\n// after: ensure complete copy, then query\naws s3 cp s3://src/data.orc s3://dst/data.orc --expected-size 123456789\nSELECT count(*) FROM t;","handlingStrategy":"try-catch","validationCode":"// Before reading, check completeness:\nlong expected = sourceFileLength(); // authoritative size from manifest/checksum\nlong actual = Files.size(path);\nif (actual < expected) throw new IllegalStateException(\"ORC file truncated: \" + actual + \" < \" + expected);","typeGuard":null,"tryCatchPattern":"try {\n    orcReader.read(...);\n} catch (PrestoException e) {\n    if (e.getCause() instanceof OrcCorruptionException && e.getMessage().contains(\"Read past end of RLE integer\")) {\n        logger.warn(\"ORC stream truncated, skipping file/stripe\");\n        // skip or re-fetch the file, then retry\n    } else {\n        throw e;\n    }\n}","preventionTips":["Verify file size/checksum against the source before querying copied files.","Never read ORC files while a writer is still appending to them.","Watch storage-layer alerts (HDFS block corruption, S3 integrity errors).","Keep Presto and the ORC writer versions compatible.","Detect truncation early by validating footers (orc-tools meta) before large queries."],"tags":["orc","data-corruption","truncated-file","presto"],"backgroundTag":"truncated-orc-stripe","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}