prestodb/presto · error · OrcCorruptionException

Dictionary is not empty but data stream is not present for %

Error message

Dictionary is not empty but data stream is not present for %s

What it means

LongDictionaryProvider.loadDictionary opens the dictionary DATA stream to build the long dictionary for a stripe. If the stream cannot be opened (openStream returns null) the stripe claims dictionary entries but has no data to read them from; the reader throws OrcCorruptionException naming the stream descriptor.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/LongDictionaryProvider.java:125

            StreamDescriptor sharedDictionaryStreamDescriptor = streamDescriptor.duplicate(DEFAULT_SEQUENCE_ID);
            InputStreamSource<LongInputStream> sharedDictionaryDataStream = dictionaryStreamSources.getInputStreamSource(sharedDictionaryStreamDescriptor, DICTIONARY_DATA, LongInputStream.class);
            long[] dictionaryBuffer = loadDictionary(streamDescriptor, sharedDictionaryDataStream, dictionary, items).dictionaryBuffer();
            sharedDictionary = new SharedDictionary(dictionaryBuffer, items);
            sharedDictionaries.put(streamId, sharedDictionary);
        }

        checkState(sharedDictionary.size == items, "Shared dictionary size mismatch for stream: %s", streamDescriptor);
        return new DictionaryResult(sharedDictionary.values, isNewEntry);
    }

    private DictionaryResult loadDictionary(StreamDescriptor streamDescriptor, InputStreamSource<LongInputStream> dictionaryDataStream, long[] dictionaryBuffer, int items)
            throws IOException
    {
        // We construct and use the input stream exactly once per stream descriptor per stripe, so we don't
        // really need to cache it.
        LongInputStream inputStream = dictionaryDataStream.openStream();
        if (inputStream == null) {
            throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), "Dictionary is not empty but data stream is not present for %s", streamDescriptor);
        }

        if (dictionaryBuffer == null || dictionaryBuffer.length < items) {
            dictionaryBuffer = new long[items];
        }

        inputStream.next(dictionaryBuffer, items);
        return new DictionaryResult(dictionaryBuffer, true);
    }
}

View on GitHub (pinned to 55bb57d202)

Solutions

1. Validate with orc-tools verify and rewrite/repair the affected files
2. Restore the partition from a good copy
3. Fix/upgrade the writer that omitted the dictionary data stream
4. Upgrade Presto if a known writer-version compatibility issue applies

// orc-tools meta data.orc -- confirm dictionary DATA stream present for dictionary-encoded columns

try { ... } catch (OrcCorruptionException e) { failQueryWithDataSourceId(e.getOrcDataSourceId()); }

When it happens

Trigger: loadDictionary (called from getDictionary/dictionaryBuffer during stripe start or first read) with dictionary item count > 0 but dictionaryDataStream.openStream() == null.

Common situations: Corrupt/truncated ORC files missing the dictionary DATA stream; buggy or third-party writers; partially uploaded S3 objects.

Related errors

• Value is not null but data stream is not present
• Value is not null but data stream is missing
• Value is not null but decimal stream is not present
• Value is not null but scale stream is not present
• Value is not null but data stream is not present

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/9ddf15024cbebaac. Report an issue: GitHub.