{"record":{"id":"530f7103b3e180f7","repo":"prestodb/presto","slug":"unexpected-end-of-stream-530f71","errorCode":null,"errorMessage":"Unexpected end of stream","messagePattern":"Unexpected end of stream","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/stream/OrcInputStream.java","lineNumber":171,"sourceCode":"        if (available() == 0) {\n            advance();\n            if (buffer == null) {\n                return -1;\n            }\n        }\n        length = Math.min(length, available());\n        System.arraycopy(buffer, position, b, off, length);\n        position += length;\n        return length;\n    }\n\n    public void skipFully(long length)\n            throws IOException\n    {\n        while (length > 0) {\n            long result = skip(length);\n            if (result < 0) {\n                throw new OrcCorruptionException(orcDataSourceId, \"Unexpected end of stream\");\n            }\n            length -= result;\n        }\n    }\n\n    public void readFully(byte[] buffer, int offset, int length)\n            throws IOException\n    {\n        while (offset < length) {\n            int result = read(buffer, offset, length - offset);\n            if (result < 0) {\n                throw new OrcCorruptionException(orcDataSourceId, \"Unexpected end of stream\");\n            }\n            offset += result;\n        }\n    }\n\n    public OrcDataSourceId getOrcDataSourceId()","sourceCodeStart":153,"sourceCodeEnd":189,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/stream/OrcInputStream.java#L153-L189","documentation":"OrcInputStream.skipFully skips `length` bytes in a loop; if the underlying data source returns a negative result (EOF) before all bytes are skipped, it throws OrcCorruptionException. This means the ORC file's data ended before the metadata/index said it should — the file is truncated or the stream layout is corrupt.","triggerScenarios":"skipFully called while repositioning a value stream past data; underlying OrcDataSource has no more bytes but more skipping remains; typically a truncated file or wrong stripe/chunk offset.","commonSituations":"Incomplete ORC file upload or copy (S3/HDFS partial read); reader metadata points beyond actual data; wrong file used with cached stripe/footer metadata; network read cut short.","solutions":["Verify the file size and completeness (compare checksums, ensure writer finished/committed).","Re-read the file with fresh metadata (clear caches) in case offsets came from a stale/other file.","Check for truncated downloads/uploads; re-transfer the file.","Check that compression/chunk boundaries are not being skipped with wrong offsets (reader/writer version mismatch)."],"exampleFix":"// before: reading truncated file copied from HDFS\n// after: verify integrity before opening\nlong srcLen = orcDataSource.getEstimatedSize();\nif (srcLen < expectedFooterEnd) {\n    throw new IOException(\"ORC file truncated: \" + orcDataSourceId);\n}","handlingStrategy":"validation","validationCode":"long expected = footerMetadataEndOffset;\nlong actual = orcDataSource.getLength();\nif (actual < expected) throw new IOException(\"ORC truncated: \" + orcDataSourceId);","typeGuard":null,"tryCatchPattern":"try { stream.skipFully(len); } catch (OrcCorruptionException e) {\n    log.error(\"ORC stream truncated for %s\", e.getOrcDataSourceId(), e);\n    throw new DataReadException(\"source file incomplete\", e);\n}","preventionTips":["Verify file checksums after copy/upload before querying.","Ensure writers complete/commit files before readers open them (atomic rename in HDFS/S3).","Clear stale metadata caches when files are replaced.","Validate files with orc-tools periodically."],"tags":["orc","corruption","io","truncated-file"],"backgroundTag":"unexpected-eof-orc-stream","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}