prestodb/presto · error · OrcCorruptionException

Dictionary is not empty but dictionary length stream is not

Error message

Dictionary is not empty but dictionary length stream is not present

What it means

Same corruption check as SliceDictionaryBatchStreamReader but in the selective reader: when the stripe dictionary is non-empty, its DICTIONARY_LENGTH stream must exist. A null stream means the file metadata and streams are inconsistent and decoding cannot proceed.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDictionarySelectiveReader.java:490

    private void openRowGroup()
            throws IOException
    {
        presentStream = presentStreamSource.openStream();
        filter = context.getFilter(presentStream);

        // read the dictionary
        if (!stripeDictionaryOpen) {
            if (stripeDictionarySize > 0) {
                // resize the dictionary lengths array if necessary
                if (stripeDictionaryLength.length < stripeDictionarySize) {
                    stripeDictionaryLength = new int[stripeDictionarySize];
                }

                // read the lengths
                LongInputStream lengthStream = stripeDictionaryLengthStreamSource.openStream();
                if (lengthStream == null) {
                    throw new OrcCorruptionException(context.getStreamDescriptor().getOrcDataSourceId(), "Dictionary is not empty but dictionary length stream is not present");
                }
                lengthStream.nextIntVector(stripeDictionarySize, stripeDictionaryLength, 0);

                long dataLength = 0;
                for (int i = 0; i < stripeDictionarySize; i++) {
                    dataLength += stripeDictionaryLength[i];
                }

                dictionaryData = ensureCapacity(dictionaryData, toIntExact(dataLength));
                dictionaryOffsetVector = ensureCapacity(dictionaryOffsetVector, stripeDictionarySize + 2);

                // read dictionary values
                ByteArrayInputStream dictionaryDataStream = stripeDictionaryDataStreamSource.openStream();
                readDictionary(dictionaryDataStream, stripeDictionarySize, stripeDictionaryLength, 0, dictionaryData, dictionaryOffsetVector, maxCodePointCount, isCharType);
            }
            else {
                dictionaryData = EMPTY_DICTIONARY_DATA;
                dictionaryOffsetVector = EMPTY_DICTIONARY_OFFSETS;

View on GitHub (pinned to 55bb57d202)

Solutions

1. Run orc-tools meta/scan to confirm which streams are missing.
2. Regenerate the ORC file or restore it from backup.
3. Verify ingestion completed fully (no partial uploads/overwrites).
4. Check writer version compatibility; rewrite with a supported writer if needed.

// pre-flight: orc-tools meta file.orc | grep DICTIONARY_LENGTH  (must exist for dictionary-encoded stripes)

try {
    return reader.read();
} catch (OrcCorruptionException e) {
    log.error("Missing dictionary length stream in %s", e.getOrcDataSourceId());
    throw new DataIntegrityException("ORC dictionary metadata inconsistent", e);
}

When it happens

Trigger: read() causing openRowGroup() on a dictionary-encoded string column where stripeDictionarySize > 0 but stripeDictionaryLengthStreamSource.openStream() returns null.

Common situations: Truncated or partially uploaded ORC files; files from buggy writers that skip the dictionary length stream; corrupted replicas on distributed storage.

Related errors

• Value is not null but data stream is not present
• Dictionary is not empty but dictionary length stream is not
• Value is not null but data stream is not present
• Value is not null but data stream is missing
• Value is not null but decimal stream is not present

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/1acd0592cb35034d. Report an issue: GitHub.