prestodb/presto · error · OrcCorruptionException

Value is not null but data stream is not present

Error message

Value is not null but data stream is not present

What it means

FloatBatchStreamReader.readBlock throws this when skipping to readOffset non-null values: the FLOAT column's data stream is required for the skip but is absent from the stripe. Like the double variant, missing DATA stream for a non-null value range indicates a corrupt or miswritten file.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/FloatBatchStreamReader.java:97

    }

    @Override
    public Block readBlock()
            throws IOException
    {
        if (!rowGroupOpen) {
            openRowGroup();
        }

        if (readOffset > 0) {
            if (presentStream != null) {
                // skip ahead the present bit reader, but count the set bits
                // and use this as the skip size for the data reader
                readOffset = presentStream.countBitsSet(readOffset);
            }
            if (readOffset > 0) {
                if (dataStream == null) {
                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is not present");
                }
                dataStream.skip(readOffset);
            }
        }

        if (dataStream == null && presentStream != null) {
            presentStream.skip(nextBatchSize);
            Block nullValueBlock = RunLengthEncodedBlock.create(type, null, nextBatchSize);
            readOffset = 0;
            nextBatchSize = 0;
            return nullValueBlock;
        }

        BlockBuilder builder = type.createBlockBuilder(null, nextBatchSize);
        if (presentStream == null) {
            if (dataStream == null) {
                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is not present");
            }

View on GitHub (pinned to 55bb57d202)

Solutions

1. Inspect stripe streams via orc-tools meta dump.
2. Rewrite/re-export the file with a compliant ORC writer.
3. Fix the writer to always produce DATA streams for populated row groups.
4. Validate file integrity (checksum, row counts) at write/ingest time.

Example fix

// before
// assume file is fine
// after
// fail fast at ingestion
if (!validateOrcStripes(file)) {
    throw new IOException("ORC file failed stripe validation; do not use: " + file);
}

// Check float column streams before reading
if (!stripeHasStream(file, floatColumnId, StreamKind.DATA)) {
    throw new IllegalStateException("Float column missing DATA stream in " + file);
}

try {
    pageSource.getNextPage();
} catch (OrcCorruptionException e) {
    metrics.increment("orc.corrupt.float.data");
    throw new UnrecoverableDataSourceException("Regenerate ORC file", e);
}

When it happens

Trigger: readBlock performs a seek where presentStream.countBitsSet(readOffset) > 0 but stripe metadata contains no DATA stream for the float column.

Common situations: Corrupt or truncated ORC files; writers that omit float DATA streams; null-count/stream-manifest inconsistencies; unsupported writer versions.

Related errors

• Value is not null but data stream is not present
• Value is not null but data stream is missing
• Value is not null but decimal stream is not present
• Value is not null but scale stream is not present
• Value is not null but data stream is not present

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/911ad26979c21853. Report an issue: GitHub.