{"record":{"id":"9c97cc43e360a439","repo":"prestodb/presto","slug":"value-is-not-null-but-data-stream-is-missing","errorCode":null,"errorMessage":"Value is not null but data stream is missing","messagePattern":"Value is not null but data stream is missing","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/reader/ByteBatchStreamReader.java","lineNumber":104,"sourceCode":"    }\n\n    @Override\n    public Block readBlock()\n            throws IOException\n    {\n        if (!rowGroupOpen) {\n            openRowGroup();\n        }\n\n        if (readOffset > 0) {\n            if (presentStream != null) {\n                // skip ahead the present bit reader, but count the set bits\n                // and use this as the skip size for the data reader\n                readOffset = presentStream.countBitsSet(readOffset);\n            }\n            if (readOffset > 0) {\n                if (dataStream == null) {\n                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but data stream is missing\");\n                }\n                dataStream.skip(readOffset);\n            }\n        }\n\n        Block block;\n        if (dataStream == null) {\n            if (presentStream == null) {\n                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is null but present stream is missing\");\n            }\n            presentStream.skip(nextBatchSize);\n            block = RunLengthEncodedBlock.create(TINYINT, null, nextBatchSize);\n        }\n        else if (presentStream == null) {\n            block = readNonNullBlock();\n        }\n        else {\n            boolean[] isNull = new boolean[nextBatchSize];","sourceCodeStart":86,"sourceCodeEnd":122,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/reader/ByteBatchStreamReader.java#L86-L122","documentation":"ByteBatchStreamReader.readBlock throws OrcCorruptionException when the present stream marks non-null byte values (readOffset > 0 after countBitsSet) but the byte DATA stream is missing. A TINYINT/BYTE column with non-null values must have its data stream present per the ORC spec.","triggerScenarios":"Positional skip on a BYTE/TINYINT column where set bits exist in the present stream but dataStream is null; equivalently any read hitting a stripe whose data stream was never written.","commonSituations":"Truncated ORC files from crashed/incomplete writes; incomplete replication or copy operations; writer bugs omitting data streams for tinyint columns.","solutions":["Regenerate or re-copy the ORC file from a verified source.","Run orc-tools scan to confirm the missing data stream in the affected stripe.","Ensure writers flush and close files atomically (write to temp then commit).","Catch OrcCorruptionException and mark the split/table as corrupt for remediation."],"exampleFix":"// before\nSELECT tiny_col FROM broken_table; // throws OrcCorruptionException\n// after\n-- recreate table from source data\nINSERT OVERWRITE TABLE broken_table SELECT tiny_col FROM raw_source;","handlingStrategy":"try-catch","validationCode":"// scan the file before loading: orc-tools scan file.orc","typeGuard":null,"tryCatchPattern":"try { block = streamReader.readBlock(); } catch (OrcCorruptionException e) { retryWithFreshCopyOrSkipSplit(e); }","preventionTips":["Atomic write-and-commit pattern for ORC files","Checksum verification on all file transfers","Monitor writer job completion before exposing files to queries"],"tags":["orc","corruption","byte","missing-stream","presto"],"backgroundTag":"orc-corruption-missing-stream","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-12T02:17:10.037Z"}