{"record":{"id":"6613834ee34a03bd","repo":"prestodb/presto","slug":"value-is-not-null-but-data-stream-is-not-present-661383","errorCode":null,"errorMessage":"Value is not null but data stream is not present","messagePattern":"Value is not null but data stream is not present","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/reader/DoubleBatchStreamReader.java","lineNumber":94,"sourceCode":"    }\n\n    @Override\n    public Block readBlock()\n            throws IOException\n    {\n        if (!rowGroupOpen) {\n            openRowGroup();\n        }\n\n        if (readOffset > 0) {\n            if (presentStream != null) {\n                // skip ahead the present bit reader, but count the set bits\n                // and use this as the skip size for the data reader\n                readOffset = presentStream.countBitsSet(readOffset);\n            }\n            if (readOffset > 0) {\n                if (dataStream == null) {\n                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but data stream is not present\");\n                }\n                dataStream.skip(readOffset);\n            }\n        }\n\n        if (dataStream == null && presentStream != null) {\n            presentStream.skip(nextBatchSize);\n            Block nullValueBlock = RunLengthEncodedBlock.create(DOUBLE, null, nextBatchSize);\n            readOffset = 0;\n            nextBatchSize = 0;\n            return nullValueBlock;\n        }\n\n        BlockBuilder builder = DOUBLE.createBlockBuilder(null, nextBatchSize);\n        if (presentStream == null) {\n            if (dataStream == null) {\n                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but data stream is not present\");\n            }","sourceCodeStart":76,"sourceCodeEnd":112,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/reader/DoubleBatchStreamReader.java#L76-L112","documentation":"In DoubleBatchStreamReader.readBlock, when seeking into a row group the reader computes readOffset (non-null values to skip) and requires the DOUBLE data stream. If dataStream is null despite values being non-null, the file is declared corrupt: a double column with values must have a DATA stream.","triggerScenarios":"readBlock called with a prior read offset (row-group seek) where presentStream.countBitsSet(readOffset) > 0 but the stripe contains no DATA stream for the double column.","commonSituations":"Truncated/corrupted ORC files; buggy third-party writers omitting the DATA stream; disagreement between present stream null counts and stripe stream list; files transferred incompletely.","solutions":["Dump file metadata with orc-tools to verify stream presence for the double column.","Re-export/rewrite the data with a trusted writer and re-read.","Fix the producing writer to always emit DATA streams for non-empty row groups.","Validate files at ingestion time (row count / checksum) before querying."],"exampleFix":"// before\npageSource.getNextPage(); // OrcCorruptionException on corrupt file\n// after\ntry {\n    pageSource.getNextPage();\n}\ncatch (OrcCorruptionException e) {\n    log.error(\"ORC file %s is corrupt: %s\", dataSourceId, e.getMessage());\n    throw new DataRepairRequiredException(\"Rewrite file from source of truth\", e);\n}","handlingStrategy":"try-catch","validationCode":"// Validate the file once before opening readers\nif (!hasRequiredStreams(file, columnId, StreamKind.DATA)) {\n    throw new IllegalStateException(\"Double column missing DATA stream: \" + file);\n}","typeGuard":null,"tryCatchPattern":"try {\n    pageSource.getNextPage();\n} catch (OrcCorruptionException e) {\n    log.error(\"Corrupt ORC file %s: %s\", file, e.getMessage());\n    quarantineFile(file);\n    throw e;\n}","preventionTips":["Check stripe stream lists with orc-tools when onboarding new producers.","Regenerate files from source of truth after corruption detection.","Keep null-mask and data stream writing consistent in custom writers.","Validate row counts per stripe at ingestion."],"tags":["orc","corruption","double","missing-stream"],"backgroundTag":"orc-corrupt-file","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}