{"record":{"id":"344c7a5d4c513f99","repo":"prestodb/presto","slug":"unexpected-end-of-stream","errorCode":null,"errorMessage":"Unexpected end of stream","messagePattern":"Unexpected end of stream","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/stream/ByteInputStream.java","lineNumber":139,"sourceCode":"\n    public byte[] next(int items)\n            throws IOException\n    {\n        byte[] values = new byte[items];\n        next(values, items);\n        return values;\n    }\n\n    public void next(byte[] values, int items)\n            throws IOException\n    {\n        int outputOffset = 0;\n        while (outputOffset < items) {\n            if (offset == length) {\n                readNextBlock();\n            }\n            if (length == 0) {\n                throw new OrcCorruptionException(input.getOrcDataSourceId(), \"Unexpected end of stream\");\n            }\n\n            int chunkSize = min(items - outputOffset, length - offset);\n            System.arraycopy(buffer, offset, values, outputOffset, chunkSize);\n\n            outputOffset += chunkSize;\n            offset += chunkSize;\n        }\n    }\n}\n","sourceCodeStart":121,"sourceCodeEnd":150,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/stream/ByteInputStream.java#L121-L150","documentation":"Thrown by ByteInputStream.next when, after readNextBlock, the internal buffer length is still 0 while the caller still needs more values — i.e. the stream is exhausted but the decoder expects `items` more bytes. It protects the reader from silently returning garbage when the byte stream ends prematurely.","triggerScenarios":"Public next(values, items) loops copying bytes out of the RLE buffer; readNextBlock() sets offset=0 and on EOF leaves length==0, so the loop guard detects no data and throws before any caller (e.g. a Boolean/Byte stream reader) consumes more items than the stream holds.","commonSituations":"Row-group metadata declares more non-null values than the stream encodes; files truncated mid-strip; ORC files written by a buggy/older writer with wrong statistics; corruption during upload/storage.","solutions":["Run orc-tools meta to compare declared stream lengths with actual data; treat mismatch as proof of corruption.","Re-copy or regenerate the ORC file from the source data.","Check for mismatched reader/writer versions and upgrade Presto to a release that matches the file's ORC spec version.","Verify storage integrity (checksums, ETags) and re-enable them if disabled.","Narrow the query (e.g. read earlier row groups) to isolate which stripe is corrupt, then repair only that file."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// cheap pre-check: does the file end past the last stripe offset?\nlong footerEnd = f.length();\nlong lastStripeEnd = readLastStripeOffsetFromFooter(f);\nif (lastStripeEnd >= footerEnd) { throw new IllegalStateException(\"stripes extend past EOF\"); }","typeGuard":null,"tryCatchPattern":"try {\n    return reader.readBlock(column);\n} catch (OrcCorruptionException e) {\n    if (e.getMessage().contains(\"Unexpected end of stream\")) {\n        return fallback.readFromRebuiltCopy(path);\n    }\n    throw e;\n}","preventionTips":["Validate with orc-dump that stripe stream lengths match actual bytes.","Regenerate rather than repair corrupt files in place.","Track corruption per source bucket/writer version to find the producer at fault.","Keep reader and writer versions compatible."],"tags":["orc","corruption","eof","truncated-file"],"backgroundTag":"orc-file-corruption","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}