prestodb/presto · error · OrcCorruptionException

Value is not null but data stream is missing

Error message

Value is not null but data stream is missing

What it means

ByteBatchStreamReader.readBlock throws OrcCorruptionException when the present stream marks non-null byte values (readOffset > 0 after countBitsSet) but the byte DATA stream is missing. A TINYINT/BYTE column with non-null values must have its data stream present per the ORC spec.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/ByteBatchStreamReader.java:104

    }

    @Override
    public Block readBlock()
            throws IOException
    {
        if (!rowGroupOpen) {
            openRowGroup();
        }

        if (readOffset > 0) {
            if (presentStream != null) {
                // skip ahead the present bit reader, but count the set bits
                // and use this as the skip size for the data reader
                readOffset = presentStream.countBitsSet(readOffset);
            }
            if (readOffset > 0) {
                if (dataStream == null) {
                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is not null but data stream is missing");
                }
                dataStream.skip(readOffset);
            }
        }

        Block block;
        if (dataStream == null) {
            if (presentStream == null) {
                throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Value is null but present stream is missing");
            }
            presentStream.skip(nextBatchSize);
            block = RunLengthEncodedBlock.create(TINYINT, null, nextBatchSize);
        }
        else if (presentStream == null) {
            block = readNonNullBlock();
        }
        else {
            boolean[] isNull = new boolean[nextBatchSize];

View on GitHub (pinned to 55bb57d202)

Solutions

1. Regenerate or re-copy the ORC file from a verified source.
2. Run orc-tools scan to confirm the missing data stream in the affected stripe.
3. Ensure writers flush and close files atomically (write to temp then commit).
4. Catch OrcCorruptionException and mark the split/table as corrupt for remediation.

Example fix

// before
SELECT tiny_col FROM broken_table; // throws OrcCorruptionException
// after
-- recreate table from source data
INSERT OVERWRITE TABLE broken_table SELECT tiny_col FROM raw_source;

// scan the file before loading: orc-tools scan file.orc

try { block = streamReader.readBlock(); } catch (OrcCorruptionException e) { retryWithFreshCopyOrSkipSplit(e); }

When it happens

Trigger: Positional skip on a BYTE/TINYINT column where set bits exist in the present stream but dataStream is null; equivalently any read hitting a stripe whose data stream was never written.

Common situations: Truncated ORC files from crashed/incomplete writes; incomplete replication or copy operations; writer bugs omitting data streams for tinyint columns.

Related errors

• Value is not null but data stream is not present
• Value is null but present stream is missing
• Value is not null but decimal stream is not present
• Value is not null but scale stream is not present
• Value is null but present stream is missing

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/9c97cc43e360a439. Report an issue: GitHub.