{"record":{"id":"9ddf15024cbebaac","repo":"prestodb/presto","slug":"dictionary-is-not-empty-but-data-stream-is-not-pre","errorCode":null,"errorMessage":"Dictionary is not empty but data stream is not present for %s","messagePattern":"Dictionary is not empty but data stream is not present for (.+?)","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/reader/LongDictionaryProvider.java","lineNumber":125,"sourceCode":"            StreamDescriptor sharedDictionaryStreamDescriptor = streamDescriptor.duplicate(DEFAULT_SEQUENCE_ID);\n            InputStreamSource<LongInputStream> sharedDictionaryDataStream = dictionaryStreamSources.getInputStreamSource(sharedDictionaryStreamDescriptor, DICTIONARY_DATA, LongInputStream.class);\n            long[] dictionaryBuffer = loadDictionary(streamDescriptor, sharedDictionaryDataStream, dictionary, items).dictionaryBuffer();\n            sharedDictionary = new SharedDictionary(dictionaryBuffer, items);\n            sharedDictionaries.put(streamId, sharedDictionary);\n        }\n\n        checkState(sharedDictionary.size == items, \"Shared dictionary size mismatch for stream: %s\", streamDescriptor);\n        return new DictionaryResult(sharedDictionary.values, isNewEntry);\n    }\n\n    private DictionaryResult loadDictionary(StreamDescriptor streamDescriptor, InputStreamSource<LongInputStream> dictionaryDataStream, long[] dictionaryBuffer, int items)\n            throws IOException\n    {\n        // We construct and use the input stream exactly once per stream descriptor per stripe, so we don't\n        // really need to cache it.\n        LongInputStream inputStream = dictionaryDataStream.openStream();\n        if (inputStream == null) {\n            throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Dictionary is not empty but data stream is not present for %s\", streamDescriptor);\n        }\n\n        if (dictionaryBuffer == null || dictionaryBuffer.length < items) {\n            dictionaryBuffer = new long[items];\n        }\n\n        inputStream.next(dictionaryBuffer, items);\n        return new DictionaryResult(dictionaryBuffer, true);\n    }\n}\n","sourceCodeStart":107,"sourceCodeEnd":136,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/reader/LongDictionaryProvider.java#L107-L136","documentation":"LongDictionaryProvider.loadDictionary opens the dictionary DATA stream to build the long dictionary for a stripe. If the stream cannot be opened (openStream returns null) the stripe claims dictionary entries but has no data to read them from; the reader throws OrcCorruptionException naming the stream descriptor.","triggerScenarios":"loadDictionary (called from getDictionary/dictionaryBuffer during stripe start or first read) with dictionary item count > 0 but dictionaryDataStream.openStream() == null.","commonSituations":"Corrupt/truncated ORC files missing the dictionary DATA stream; buggy or third-party writers; partially uploaded S3 objects.","solutions":["Validate with orc-tools verify and rewrite/repair the affected files","Restore the partition from a good copy","Fix/upgrade the writer that omitted the dictionary data stream","Upgrade Presto if a known writer-version compatibility issue applies"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// orc-tools meta data.orc -- confirm dictionary DATA stream present for dictionary-encoded columns","typeGuard":null,"tryCatchPattern":"try { ... } catch (OrcCorruptionException e) { failQueryWithDataSourceId(e.getOrcDataSourceId()); }","preventionTips":["Gate ingestion on orc-tools verify","Avoid partially uploaded files (use atomic commit/copy for S3/HDFS)","Fix writers that declare dictionary encodings without the dictionary stream","Restore corrupt partitions from source of truth"],"tags":["orc","corruption","dictionary","missing-stream"],"backgroundTag":"orc-corruption-missing-stream","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}