prestodb/presto · error · OrcCorruptionException

Value is not null but data stream is not present

Error message

Value is not null but data stream is not present

What it means

In DoubleBatchStreamReader.readBlock, when seeking into a row group the reader computes readOffset (non-null values to skip) and requires the DOUBLE data stream. If dataStream is null despite values being non-null, the file is declared corrupt: a double column with values must have a DATA stream.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/DoubleBatchStreamReader.java:94

    }

    @Override
    public Block readBlock()
            throws IOException
    {
        if (!rowGroupOpen) {
            openRowGroup();
        }

        if (readOffset > 0) {
            if (presentStream != null) {
                // skip ahead the present bit reader, but count the set bits
                // and use this as the skip size for the data reader
                readOffset = presentStream.countBitsSet(readOffset);
            }
            if (readOffset > 0) {
                if (dataStream == null) {
                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is not present");
                }
                dataStream.skip(readOffset);
            }
        }

        if (dataStream == null && presentStream != null) {
            presentStream.skip(nextBatchSize);
            Block nullValueBlock = RunLengthEncodedBlock.create(DOUBLE, null, nextBatchSize);
            readOffset = 0;
            nextBatchSize = 0;
            return nullValueBlock;
        }

        BlockBuilder builder = DOUBLE.createBlockBuilder(null, nextBatchSize);
        if (presentStream == null) {
            if (dataStream == null) {
                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is not present");
            }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Dump file metadata with orc-tools to verify stream presence for the double column.
2. Re-export/rewrite the data with a trusted writer and re-read.
3. Fix the producing writer to always emit DATA streams for non-empty row groups.
4. Validate files at ingestion time (row count / checksum) before querying.

Example fix

// before
pageSource.getNextPage(); // OrcCorruptionException on corrupt file
// after
try {
    pageSource.getNextPage();
}
catch (OrcCorruptionException e) {
    log.error("ORC file %s is corrupt: %s", dataSourceId, e.getMessage());
    throw new DataRepairRequiredException("Rewrite file from source of truth", e);
}

// Validate the file once before opening readers
if (!hasRequiredStreams(file, columnId, StreamKind.DATA)) {
    throw new IllegalStateException("Double column missing DATA stream: " + file);
}

try {
    pageSource.getNextPage();
} catch (OrcCorruptionException e) {
    log.error("Corrupt ORC file %s: %s", file, e.getMessage());
    quarantineFile(file);
    throw e;
}

When it happens

Trigger: readBlock called with a prior read offset (row-group seek) where presentStream.countBitsSet(readOffset) > 0 but the stripe contains no DATA stream for the double column.

Common situations: Truncated/corrupted ORC files; buggy third-party writers omitting the DATA stream; disagreement between present stream null counts and stripe stream list; files transferred incompletely.

Related errors

• Value is not null but data stream is not present
• Value is not null but data stream is missing
• Value is not null but decimal stream is not present
• Value is not null but scale stream is not present
• Value is not null but data stream is not present

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/6613834ee34a03bd. Report an issue: GitHub.