{"record":{"id":"341a8249be25e77b","repo":"apache/flink","slug":"could-not-decode-the-dictionary-for","errorCode":null,"errorMessage":"could not decode the dictionary for {}","messagePattern":"could not decode the dictionary for (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/reader/AbstractColumnReader.java","lineNumber":117,"sourceCode":"    ByteBufferInputStream dataInputStream;\n\n    /** Dictionary decoder to wrap dictionary ids input stream. */\n    private RunLengthDecoder dictionaryIdsDecoder;\n\n    public AbstractColumnReader(ColumnDescriptor descriptor, PageReader pageReader)\n            throws IOException {\n        this.descriptor = descriptor;\n        this.pageReader = pageReader;\n        this.maxDefLevel = descriptor.getMaxDefinitionLevel();\n\n        DictionaryPage dictionaryPage = pageReader.readDictionaryPage();\n        if (dictionaryPage != null) {\n            try {\n                this.dictionary =\n                        dictionaryPage.getEncoding().initDictionary(descriptor, dictionaryPage);\n                this.isCurrentPageDictionaryEncoded = true;\n            } catch (IOException e) {\n                throw new IOException(\"could not decode the dictionary for \" + descriptor, e);\n            }\n        } else {\n            this.dictionary = null;\n            this.isCurrentPageDictionaryEncoded = false;\n        }\n        /*\n         * Total number of values in this column (in this row group).\n         */\n        long totalValueCount = pageReader.getTotalValueCount();\n        if (totalValueCount == 0) {\n            throw new IOException(\"totalValueCount == 0\");\n        }\n    }\n\n    protected void checkTypeName(PrimitiveType.PrimitiveTypeName expectedName) {\n        PrimitiveType.PrimitiveTypeName actualName =\n                descriptor.getPrimitiveType().getPrimitiveTypeName();\n        checkArgument(","sourceCodeStart":99,"sourceCodeEnd":135,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/reader/AbstractColumnReader.java#L99-L135","documentation":"IOException from AbstractColumnReader.initFromPages/constructor: the column chunk carried a dictionary page, but dictionaryPage.getEncoding().initDictionary(descriptor, dictionaryPage) threw - the dictionary encoding is unsupported for this physical type or the dictionary bytes are corrupt. The original exception is chained as the cause.","triggerScenarios":"Opening a column reader for a chunk whose dictionary page uses an encoding the reader cannot decode (e.g. a newer/obsolete dictionary encoding for the physical type), or whose dictionary payload is malformed.","commonSituations":"Parquet files written by writers with encoding extensions (e.g. some Delta/ALPHA encodings) that stock parquet-mr cannot decode; bit-level corruption in dictionary pages from bad storage; version skew between writer-side and Flink's bundled parquet-mr.","solutions":["Read the cause exception to identify the failing encoding","Rewrite the file with a standard/older writer (PLAIN_DICTIONARY/RLE_DICTIONARY) or disable exotic encodings on the producer","Upgrade Flink so its bundled parquet-mr knows the encoding","Verify storage-level integrity (checksums, re-download) to rule out corruption"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (IOException e) { if (e.getMessage() != null && e.getMessage().startsWith(\"could not decode the dictionary\")) { Throwable c = e.getCause(); /* inspect encoding in c, rewrite file with standard encodings or upgrade Flink */ } else throw e; }","preventionTips":["Pin writer and reader to compatible parquet-mr versions","Avoid non-standard encoder settings (experimental delta encodings) for data Flink must read"],"tags":["parquet","encoding","dictionary","corruption","io","flink"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}