prestodb/presto · error · OrcCorruptionException

Value is not null but data stream is not present

Error message

Value is not null but data stream is not present

What it means

BooleanBatchStreamReader.readBlock throws OrcCorruptionException when a boolean column has non-null values to read (present stream indicates set bits) but the underlying boolean DATA stream is absent from the file. Per the ORC spec, any column with non-null values must carry a data stream, so this signals a truncated or malformed file.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/BooleanBatchStreamReader.java:103

    }

    @Override
    public Block readBlock()
            throws IOException
    {
        if (!rowGroupOpen) {
            openRowGroup();
        }

        if (readOffset > 0) {
            if (presentStream != null) {
                // skip ahead the present bit reader, but count the set bits
                // and use this as the skip size for the data reader
                readOffset = presentStream.countBitsSet(readOffset);
            }
            if (readOffset > 0) {
                if (dataStream == null) {
                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is not present");
                }
                dataStream.skip(readOffset);
            }
        }

        if (dataStream == null && presentStream != null) {
            presentStream.skip(nextBatchSize);
            Block nullValueBlock = RunLengthEncodedBlock.create(BOOLEAN, null, nextBatchSize);
            readOffset = 0;
            nextBatchSize = 0;
            return nullValueBlock;
        }

        Block block;
        if (dataStream == null) {
            if (presentStream == null) {
                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is null but present stream is missing");
            }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Verify file integrity and re-copy/regenerate the ORC file from the source data.
2. Use orc-tools 'meta'/'scan' to confirm missing streams in the stripe/footer.
3. Check that writers complete and close files properly (no crash mid-stripe).
4. Upgrade the ORC writer version if known bugs caused missing streams.

Example fix

// before: querying a truncated file directly
SELECT bool_col FROM corrupted_table;
// after: regenerate the file, or guard the read
try { ... reader.readBlock(...) } catch (OrcCorruptionException e) { log.error("missing data stream, regenerate file"); }

// validate file has expected streams before read
// orc-tools meta file.orc  -- confirm DATA stream exists for boolean columns

try { block = streamReader.readBlock(); } catch (OrcCorruptionException e) { markSplitFailed(e); }

When it happens

Trigger: Skipping into a BOOLEAN column (readOffset > 0 via setRowOffset) where countBitsSet found non-null positions but dataStream is null; generally any read of a BOOLEAN column whose data stream is missing while present stream marks values non-null.

Common situations: ORC files truncated mid-write (writer crashed before flushing data streams); files copied incompletely from HDFS/S3; files produced by faulty third-party ORC writers.

Related errors

• Value is null but present stream is missing
• Value is not null but data stream is missing
• Value is not null but decimal stream is not present
• Value is not null but scale stream is not present
• Value is null but present stream is missing

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/67b0c2935d74f170. Report an issue: GitHub.