{"record":{"id":"77b7f468d8fc97bc","repo":"prestodb/presto","slug":"value-is-not-null-but-data-stream-is-not-present-77b7f4","errorCode":null,"errorMessage":"Value is not null but data stream is not present","messagePattern":"Value is not null but data stream is not present","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDictionaryBatchStreamReader.java","lineNumber":137,"sourceCode":"    }\n\n    @Override\n    public Block readBlock()\n            throws IOException\n    {\n        if (!rowGroupOpen) {\n            openRowGroup();\n        }\n\n        if (readOffset > 0) {\n            if (presentStream != null) {\n                // skip ahead the present bit reader, but count the set bits\n                // and use this as the skip size for the length reader\n                readOffset = presentStream.countBitsSet(readOffset);\n            }\n            if (readOffset > 0) {\n                if (dataStream == null) {\n                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but data stream is not present\");\n                }\n                if (inDictionaryStream != null) {\n                    inDictionaryStream.skip(readOffset);\n                }\n                dataStream.skip(readOffset);\n            }\n        }\n\n        int[] idsVector = new int[nextBatchSize];\n        if (presentStream == null) {\n            // Data doesn't have nulls\n            if (dataStream == null) {\n                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but data stream is not present\");\n            }\n            if (inDictionaryStream == null) {\n                dataStream.next(idsVector, nextBatchSize);\n            }\n            else {","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDictionaryBatchStreamReader.java#L119-L155","documentation":"While skipping over non-null positions at the start of a batch, the reader found the DATA stream missing even though there are non-null values to read. This indicates the ORC row group's stream layout does not match the declared column encoding, i.e. the file metadata is corrupt.","triggerScenarios":"readBlock() with readOffset > 0 (positions were skipped via prepareNextRead) and a presentStream that marks some values as non-null, while the DATA stream for the column is absent from the row group.","commonSituations":"Truncated or partially written ORC files; files written by buggy or non-conformant writers that omit DATA streams; corruption during transfer to object storage.","solutions":["Validate the ORC file (orc-tools 'meta'/'scan') to confirm missing DATA streams.","Re-copy or regenerate the corrupt file from the source system.","Check the writer version; if written by a non-Presto/buggy writer, rewrite with a compliant writer.","Point the query at a healthy replica or backup of the file."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// pre-flight: verify file integrity before querying\n// orc-tools meta file.orc  (confirm DATA stream exists for the column)","typeGuard":null,"tryCatchPattern":"try {\n    Block block = reader.readBlock();\n} catch (OrcCorruptionException e) {\n    log.error(\"ORC file %s is corrupt: %s\", e.getOrcDataSourceId(), e.getMessage());\n    throw new DataIntegrityException(\"Fall back to a replica or re-copy the file\", e);\n}","preventionTips":["Validate ORC files with orc-tools at ingest time before exposing them to queries.","Use write-then-rename so readers never see partially written files.","Enable checksum verification on object storage copies."],"tags":["orc","corruption","missing-stream"],"backgroundTag":"orc-file-corruption","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}