prestodb/presto · error · OrcCorruptionException

Value is not null but data stream is not present

Error message

Value is not null but data stream is not present

What it means

ListBatchStreamReader.readBlock throws this (message text says 'data stream' but the guard is on lengthStream) when seeking past readOffset values: the LIST column's LENGTH stream is needed to compute the total element skip size, but it is absent. The stream carries the element counts of each list value, so without it the reader cannot know how many child elements to skip — the file is treated as corrupt.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/ListBatchStreamReader.java:106

    }

    @Override
    public Block readBlock()
            throws IOException
    {
        if (!rowGroupOpen) {
            openRowGroup();
        }

        if (readOffset > 0) {
            if (presentStream != null) {
                // skip ahead the present bit reader, but count the set bits
                // and use this as the skip size for the data reader
                readOffset = presentStream.countBitsSet(readOffset);
            }
            if (readOffset > 0) {
                if (lengthStream == null) {
                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is not present");
                }
                long elementSkipSize = lengthStream.sum(readOffset);
                elementStreamReader.prepareNextRead(toIntExact(elementSkipSize));
            }
        }

        // We will use the offsetVector as the buffer to read the length values from lengthStream,
        // and the length values will be converted in-place to an offset vector.
        int[] offsetVector = new int[nextBatchSize + 1];
        boolean[] nullVector = null;
        if (presentStream == null) {
            if (lengthStream == null) {
                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is not present");
            }
            lengthStream.next(offsetVector, nextBatchSize);
        }
        else {
            nullVector = new boolean[nextBatchSize];

View on GitHub (pinned to 55bb57d202)

Solutions

1. Dump stripe metadata with orc-tools to confirm the LENGTH stream is missing for the list column.
2. Rewrite the file with a compliant ORC writer and re-read.
3. Fix the upstream writer to always emit the LENGTH stream for list columns.
4. Validate files at ingestion (stripe stream checks) before analytics use.

Example fix

// before
// reading a corrupted file directly
try { pageSource.getNextPage(); }
// after
catch (OrcCorruptionException e) {
    metrics.corruptFileCounter.increment();
    reprocessFromSource(file, e);
}

// LIST columns always require a LENGTH stream
if (!stripeHasStream(file, listColumnId, StreamKind.LENGTH)) {
    throw new IllegalStateException("LIST column missing LENGTH stream in " + file);
}

try {
    pageSource.getNextPage();
} catch (OrcCorruptionException e) {
    log.error("List LENGTH stream missing; file corrupt: %s", e.getMessage());
    throw e;
}

When it happens

Trigger: readBlock seek path with readOffset > 0 where the stripe has no LENGTH stream for the LIST column (lengthStream == null).

Common situations: Corrupt ORC files missing the list LENGTH stream; buggy writers writing offsets/lengths incorrectly; schema/stream mismatch after partial rewrite; incompatible writer versions.

Related errors

• Value is not null but data stream is not present
• Value is not null but data stream is missing
• Value is not null but decimal stream is not present
• Value is not null but scale stream is not present
• Value is not null but data stream is not present

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/52f8725fe392f474. Report an issue: GitHub.