{"record":{"id":"1acd0592cb35034d","repo":"prestodb/presto","slug":"dictionary-is-not-empty-but-dictionary-length-stre-1acd05","errorCode":null,"errorMessage":"Dictionary is not empty but dictionary length stream is not present","messagePattern":"Dictionary is not empty but dictionary length stream is not present","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDictionarySelectiveReader.java","lineNumber":490,"sourceCode":"\n    private void openRowGroup()\n            throws IOException\n    {\n        presentStream = presentStreamSource.openStream();\n        filter = context.getFilter(presentStream);\n\n        // read the dictionary\n        if (!stripeDictionaryOpen) {\n            if (stripeDictionarySize > 0) {\n                // resize the dictionary lengths array if necessary\n                if (stripeDictionaryLength.length < stripeDictionarySize) {\n                    stripeDictionaryLength = new int[stripeDictionarySize];\n                }\n\n                // read the lengths\n                LongInputStream lengthStream = stripeDictionaryLengthStreamSource.openStream();\n                if (lengthStream == null) {\n                    throw new OrcCorruptionException(context.getStreamDescriptor().getOrcDataSourceId(), \"Dictionary is not empty but dictionary length stream is not present\");\n                }\n                lengthStream.nextIntVector(stripeDictionarySize, stripeDictionaryLength, 0);\n\n                long dataLength = 0;\n                for (int i = 0; i < stripeDictionarySize; i++) {\n                    dataLength += stripeDictionaryLength[i];\n                }\n\n                dictionaryData = ensureCapacity(dictionaryData, toIntExact(dataLength));\n                dictionaryOffsetVector = ensureCapacity(dictionaryOffsetVector, stripeDictionarySize + 2);\n\n                // read dictionary values\n                ByteArrayInputStream dictionaryDataStream = stripeDictionaryDataStreamSource.openStream();\n                readDictionary(dictionaryDataStream, stripeDictionarySize, stripeDictionaryLength, 0, dictionaryData, dictionaryOffsetVector, maxCodePointCount, isCharType);\n            }\n            else {\n                dictionaryData = EMPTY_DICTIONARY_DATA;\n                dictionaryOffsetVector = EMPTY_DICTIONARY_OFFSETS;","sourceCodeStart":472,"sourceCodeEnd":508,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDictionarySelectiveReader.java#L472-L508","documentation":"Same corruption check as SliceDictionaryBatchStreamReader but in the selective reader: when the stripe dictionary is non-empty, its DICTIONARY_LENGTH stream must exist. A null stream means the file metadata and streams are inconsistent and decoding cannot proceed.","triggerScenarios":"read() causing openRowGroup() on a dictionary-encoded string column where stripeDictionarySize > 0 but stripeDictionaryLengthStreamSource.openStream() returns null.","commonSituations":"Truncated or partially uploaded ORC files; files from buggy writers that skip the dictionary length stream; corrupted replicas on distributed storage.","solutions":["Run orc-tools meta/scan to confirm which streams are missing.","Regenerate the ORC file or restore it from backup.","Verify ingestion completed fully (no partial uploads/overwrites).","Check writer version compatibility; rewrite with a supported writer if needed."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// pre-flight: orc-tools meta file.orc | grep DICTIONARY_LENGTH  (must exist for dictionary-encoded stripes)","typeGuard":null,"tryCatchPattern":"try {\n    return reader.read();\n} catch (OrcCorruptionException e) {\n    log.error(\"Missing dictionary length stream in %s\", e.getOrcDataSourceId());\n    throw new DataIntegrityException(\"ORC dictionary metadata inconsistent\", e);\n}","preventionTips":["Validate ORC files at ingest time with orc-tools.","Ensure compaction jobs complete atomically before readers see the file.","Track and pin supported writer versions."],"tags":["orc","corruption","dictionary-encoding","missing-stream"],"backgroundTag":"orc-file-corruption","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}