{"record":{"id":"ad64a47e3803aa6e","repo":"prestodb/presto","slug":"value-is-not-null-but-data-stream-is-not-present-ad64a4","errorCode":null,"errorMessage":"Value is not null but data stream is not present","messagePattern":"Value is not null but data stream is not present","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/reader/MapDirectBatchStreamReader.java","lineNumber":114,"sourceCode":"    }\n\n    @Override\n    public Block readBlock()\n            throws IOException\n    {\n        if (!rowGroupOpen) {\n            openRowGroup();\n        }\n\n        if (readOffset > 0) {\n            if (presentStream != null) {\n                // skip ahead the present bit reader, but count the set bits\n                // and use this as the skip size for the data reader\n                readOffset = presentStream.countBitsSet(readOffset);\n            }\n            if (readOffset > 0) {\n                if (lengthStream == null) {\n                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but data stream is not present\");\n                }\n                long entrySkipSize = lengthStream.sum(readOffset);\n                keyStreamReader.prepareNextRead(toIntExact(entrySkipSize));\n                valueStreamReader.prepareNextRead(toIntExact(entrySkipSize));\n            }\n        }\n\n        // We will use the offsetVector as the buffer to read the length values from lengthStream,\n        // and the length values will be converted in-place to an offset vector.\n        int[] offsetVector = new int[nextBatchSize + 1];\n        boolean[] nullVector = null;\n\n        if (presentStream == null) {\n            if (lengthStream == null) {\n                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Value is not null but data stream is not present\");\n            }\n            lengthStream.next(offsetVector, nextBatchSize);\n        }","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/reader/MapDirectBatchStreamReader.java#L96-L132","documentation":"In MapDirectBatchStreamReader.readBlock, when a row's map value is non-null the reader must have a length stream to locate that entry's key/value span. If the row is marked present but the length (data) stream is absent, the file's stream metadata is inconsistent with its data, so the reader throws OrcCorruptionException('Value is not null but data stream is not present'). This guards against reading a physically incomplete column.","triggerScenarios":"Reading a stripe where the present stream says at least one row is non-null but no length stream exists for the map column, and readOffset > 0 requires skipping entries via lengthStream.sum(readOffset).","commonSituations":"Truncated or partially-written ORC/DWRF files (writer crashed mid-stripe); files produced by buggy third-party writers omitting the length stream; manual file splicing/merging that dropped streams; wrong column-to-stream mapping after schema evolution.","solutions":["Validate the file with orc-tools (or a full scan of the column) to confirm the length stream is missing in the footer/metadata.","Regenerate or re-write the file with a known-good writer; replace the corrupted copy from a backup.","Check the writer version that produced the file for known bugs emitting maps without length streams.","If the corruption is expected, filter/repartition around the bad file and exclude it from the table."],"exampleFix":"// before — trusting the file blindly\nreader.readBlock(mapType, positions);\n// after — pre-validate with a checksum/validation pass\nboolean ok = new OrcValidator(orcDataSource).validate();\nif (!ok) { skipFile(); } else { reader.readBlock(mapType, positions); }","handlingStrategy":"validation","validationCode":"// before reading, verify the column has the required streams\nList<StreamId> streams = getStreamsForColumn(columnId);\nboolean hasLength = streams.stream().anyMatch(s -> s.getStreamKind() == StreamKind.LENGTH);\nboolean hasPresent = streams.stream().anyMatch(s -> s.getStreamKind() == StreamKind.PRESENT);\nif (!hasLength && !hasPresent) {\n    throw new PrestoException(ORC_BAD_DATA, \"Map column missing LENGTH stream in stripe\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    block = mapReader.readBlock(mapType, positions);\n}\ncatch (OrcCorruptionException e) {\n    throw new PrestoException(ORC_BAD_DATA, \"Corrupted map column (missing length stream): \" + e.getMessage(), e);\n}","preventionTips":["Validate ORC/DWRF files (orc-tools check) before ingesting from new producers.","Watch for truncated/partial writes; enable writer-side fsync/checksums.","Re-write files that fail footer validation instead of skipping rows.","Track producer writer versions known to emit incomplete map streams."],"tags":["orc-corruption","map","missing-stream"],"backgroundTag":"orc-file-corruption","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}