{"record":{"id":"2ebe08fc6737368b","repo":"prestodb/presto","slug":"dictionary-is-not-empty-but-dictionary-length-stre","errorCode":null,"errorMessage":"Dictionary is not empty but dictionary length stream is not present","messagePattern":"Dictionary is not empty but dictionary length stream is not present","errorType":"exception","errorClass":"OrcCorruptionException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDictionaryBatchStreamReader.java","lineNumber":228,"sourceCode":"        }\n    }\n\n    private void openRowGroup()\n            throws IOException\n    {\n        // read the dictionary\n        if (!stripeDictionaryOpen) {\n            if (stripeDictionarySize > 0) {\n                // resize the dictionary lengths array if necessary\n                if (stripeDictionaryLength.length < stripeDictionarySize) {\n                    stripeDictionaryLength = new int[stripeDictionarySize];\n                    systemMemoryContext.setBytes(sizeOf(stripeDictionaryLength));\n                }\n\n                // read the lengths\n                LongInputStream lengthStream = stripeDictionaryLengthStreamSource.openStream();\n                if (lengthStream == null) {\n                    throw new OrcCorruptionException(streamDescriptor.getOrcDataSourceId(), \"Dictionary is not empty but dictionary length stream is not present\");\n                }\n                lengthStream.next(stripeDictionaryLength, stripeDictionarySize);\n\n                long dataLength = 0;\n                for (int i = 0; i < stripeDictionarySize; i++) {\n                    dataLength += stripeDictionaryLength[i];\n                }\n\n                // we must always create a new dictionary array because the previous dictionary may still be referenced\n                stripeDictionaryData = new byte[toIntExact(dataLength)];\n                systemMemoryContext.setBytes(sizeOf(stripeDictionaryData));\n\n                // add one extra entry for null\n                stripeDictionaryOffsetVector = new int[stripeDictionarySize + 2];\n                systemMemoryContext.setBytes(sizeOf(stripeDictionaryOffsetVector));\n\n                // read dictionary values\n                ByteArrayInputStream dictionaryDataStream = stripeDictionaryDataStreamSource.openStream();","sourceCodeStart":210,"sourceCodeEnd":246,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/reader/SliceDictionaryBatchStreamReader.java#L210-L246","documentation":"During openRowGroup, the reader found stripeDictionarySize > 0 (the stripe dictionary holds entries) but the DICTIONARY_LENGTH stream is absent, so entry lengths cannot be decoded. The file's stream layout contradicts its metadata, so it is flagged as corrupt.","triggerScenarios":"readBlock() triggering openRowGroup() on a dictionary-encoded string column with stripeDictionarySize > 0 while stripeDictionaryLengthStreamSource.openStream() returns null.","commonSituations":"Truncated ORC files missing tail streams; files written by buggy writers that emit dictionary data without lengths; corruption during file transfer or partial overwrite by concurrent writers.","solutions":["Validate the ORC file with orc-tools to confirm the missing DICTIONARY_LENGTH stream.","Regenerate the file or restore from backup.","Ensure no concurrent writers overwrite the file while it is being read (write-then-rename pattern).","Check the producing writer version for known dictionary-encoding bugs."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// pre-flight: confirm DICTIONARY_LENGTH stream exists when dictionary encoding is used\n// orc-tools meta file.orc | grep DICTIONARY_LENGTH","typeGuard":null,"tryCatchPattern":"try {\n    return reader.readBlock();\n} catch (OrcCorruptionException e) {\n    log.error(\"Dictionary length stream missing in %s\", e.getOrcDataSourceId());\n    throw new DataIntegrityException(\"ORC dictionary encoding incomplete\", e);\n}","preventionTips":["Use atomic publish (write to temp path, rename) for ORC output.","Check writer version for dictionary-encoding bugs before adopting files.","Validate files at ingest with orc-tools meta."],"tags":["orc","corruption","dictionary-encoding","missing-stream"],"backgroundTag":"orc-file-corruption","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}