{"record":{"id":"f12ddee08470aa6c","repo":"prestodb/presto","slug":"value-is-not-null-but-data-stream-is-not-present-f12dde","errorCode":null,"errorMessage":"Value is not null but data stream is not present","messagePattern":"Value is not null but data stream is not present","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDirectBatchStreamReader.java","lineNumber":120,"sourceCode":"    {\n        if (!rowGroupOpen) {\n            openRowGroup();\n        }\n\n        if (readOffset > 0) {\n            if (presentStream != null) {\n                // skip ahead the present bit reader, but count the set bits\n                // and use this as the skip size for the length reader\n                readOffset = presentStream.countBitsSet(readOffset);\n            }\n            if (readOffset > 0) {\n                if (lengthStream == null) {\n                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but length stream is not present\");\n                }\n                long dataSkipSize = lengthStream.sum(readOffset);\n                if (dataSkipSize > 0) {\n                    if (dataStream == null) {\n                        throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but data stream is not present\");\n                    }\n                    dataStream.skip(dataSkipSize);\n                }\n            }\n        }\n\n        if (lengthStream == null) {\n            if (presentStream == null) {\n                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is null but present stream is missing\");\n            }\n            presentStream.skip(nextBatchSize);\n            Block nullValueBlock = readAllNullsBlock();\n            readOffset = 0;\n            nextBatchSize = 0;\n            return nullValueBlock;\n        }\n\n        // create new isNullVector and offsetVector for VariableWidthBlock","sourceCodeStart":102,"sourceCodeEnd":138,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDirectBatchStreamReader.java#L102-L138","documentation":"When skipping non-null positions, the reader computes how many data bytes to skip from the LENGTH stream; if that sum is positive, non-null data must exist, so a missing DATA stream indicates the ORC file is corrupt.","triggerScenarios":"readBlock() with readOffset > 0 where lengthStream.sum(readOffset) > 0 but dataStream == null for the column in the current row group.","commonSituations":"ORC files with truncated DATA streams (e.g. partial uploads); writers that omit DATA while still writing lengths; storage corruption or manual file edits.","solutions":["Validate the ORC file with orc-tools to confirm the missing DATA stream.","Regenerate or restore the file from a healthy source.","Check the ingestion pipeline for interrupted or partial uploads.","Confirm the producing writer version is supported and compliant."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// pre-flight: orc-tools scan file.orc to detect missing DATA streams across all row groups","typeGuard":null,"tryCatchPattern":"try {\n    return reader.readBlock();\n} catch (OrcCorruptionException e) {\n    log.error(\"Missing DATA stream despite non-zero length in %s\", e.getOrcDataSourceId());\n    throw new DataIntegrityException(\"ORC direct encoding incomplete\", e);\n}","preventionTips":["Validate files with orc-tools scan before serving queries.","Monitor ingestion for interrupted or partial uploads.","Keep a backup/source copy so corrupt files can be regenerated."],"tags":["orc","corruption","missing-stream"],"backgroundTag":"orc-file-corruption","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}