What causes "Value is not null but data stream is not present" in prestodb/presto?

Validate the ORC file (orc-tools 'meta'/'scan') to confirm missing DATA streams.

prestodb/presto · error · OrcCorruptionException

Value is not null but data stream is not present

Error message

Value is not null but data stream is not present

What it means

While skipping over non-null positions at the start of a batch, the reader found the DATA stream missing even though there are non-null values to read. This indicates the ORC row group's stream layout does not match the declared column encoding, i.e. the file metadata is corrupt.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDictionaryBatchStreamReader.java:137

    }

    @Override
    public Block readBlock()
            throws IOException
    {
        if (!rowGroupOpen) {
            openRowGroup();
        }

        if (readOffset > 0) {
            if (presentStream != null) {
                // skip ahead the present bit reader, but count the set bits
                // and use this as the skip size for the length reader
                readOffset = presentStream.countBitsSet(readOffset);
            }
            if (readOffset > 0) {
                if (dataStream == null) {
                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is not present");
                }
                if (inDictionaryStream != null) {
                    inDictionaryStream.skip(readOffset);
                }
                dataStream.skip(readOffset);
            }
        }

        int[] idsVector = new int[nextBatchSize];
        if (presentStream == null) {
            // Data doesn't have nulls
            if (dataStream == null) {
                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is not present");
            }
            if (inDictionaryStream == null) {
                dataStream.next(idsVector, nextBatchSize);
            }
            else {

View on GitHub (pinned to 55bb57d202)

Solutions

Validate the ORC file (orc-tools 'meta'/'scan') to confirm missing DATA streams.
Re-copy or regenerate the corrupt file from the source system.
Check the writer version; if written by a non-Presto/buggy writer, rewrite with a compliant writer.
Point the query at a healthy replica or backup of the file.

Defensive patterns

Strategy: try-catch

Validate before calling

// pre-flight: verify file integrity before querying
// orc-tools meta file.orc  (confirm DATA stream exists for the column)

Try / catch

try {
    Block block = reader.readBlock();
} catch (OrcCorruptionException e) {
    log.error("ORC file %s is corrupt: %s", e.getOrcDataSourceId(), e.getMessage());
    throw new DataIntegrityException("Fall back to a replica or re-copy the file", e);
}

Prevention

Validate ORC files with orc-tools at ingest time before exposing them to queries.
Use write-then-rename so readers never see partially written files.
Enable checksum verification on object storage copies.

When it happens

Trigger: readBlock() with readOffset > 0 (positions were skipped via prepareNextRead) and a presentStream that marks some values as non-null, while the DATA stream for the column is absent from the row group.

Common situations: Truncated or partially written ORC files; files written by buggy or non-conformant writers that omit DATA streams; corruption during transfer to object storage.

Related errors

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/77b7f468d8fc97bc. Report an issue: GitHub.