apache/flink · error · IOException

could not decode the dictionary for

Error message

could not decode the dictionary for {}

What it means

IOException from AbstractColumnReader.initFromPages/constructor: the column chunk carried a dictionary page, but dictionaryPage.getEncoding().initDictionary(descriptor, dictionaryPage) threw - the dictionary encoding is unsupported for this physical type or the dictionary bytes are corrupt. The original exception is chained as the cause.

Solutions

1. Read the cause exception to identify the failing encoding
2. Rewrite the file with a standard/older writer (PLAIN_DICTIONARY/RLE_DICTIONARY) or disable exotic encodings on the producer
3. Upgrade Flink so its bundled parquet-mr knows the encoding
4. Verify storage-level integrity (checksums, re-download) to rule out corruption

catch (IOException e) { if (e.getMessage() != null && e.getMessage().startsWith("could not decode the dictionary")) { Throwable c = e.getCause(); /* inspect encoding in c, rewrite file with standard encodings or upgrade Flink */ } else throw e; }

When it happens

Trigger: Opening a column reader for a chunk whose dictionary page uses an encoding the reader cannot decode (e.g. a newer/obsolete dictionary encoding for the physical type), or whose dictionary payload is malformed.

Common situations: Parquet files written by writers with encoding extensions (e.g. some Delta/ALPHA encodings) that stock parquet-mr cannot decode; bit-level corruption in dictionary pages from bad storage; version skew between writer-side and Flink's bundled parquet-mr.

Related errors

• could not read page in col
• expecting more rows but reached last block. Read
• totalValueCount == 0
• Corrupted Parquet schema
• Error while waiting for job to be initialized

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/341a8249be25e77b. Report an issue: GitHub.

Appendix: source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/reader/AbstractColumnReader.java:117

    ByteBufferInputStream dataInputStream;

    /** Dictionary decoder to wrap dictionary ids input stream. */
    private RunLengthDecoder dictionaryIdsDecoder;

    public AbstractColumnReader(ColumnDescriptor descriptor, PageReader pageReader)
            throws IOException {
        this.descriptor = descriptor;
        this.pageReader = pageReader;
        this.maxDefLevel = descriptor.getMaxDefinitionLevel();

        DictionaryPage dictionaryPage = pageReader.readDictionaryPage();
        if (dictionaryPage != null) {
            try {
                this.dictionary =
                        dictionaryPage.getEncoding().initDictionary(descriptor, dictionaryPage);
                this.isCurrentPageDictionaryEncoded = true;
            } catch (IOException e) {
                throw new IOException("could not decode the dictionary for " + descriptor, e);
            }
        } else {
            this.dictionary = null;
            this.isCurrentPageDictionaryEncoded = false;
        }
        /*
         * Total number of values in this column (in this row group).
         */
        long totalValueCount = pageReader.getTotalValueCount();
        if (totalValueCount == 0) {
            throw new IOException("totalValueCount == 0");
        }
    }

    protected void checkTypeName(PrimitiveType.PrimitiveTypeName expectedName) {
        PrimitiveType.PrimitiveTypeName actualName =
                descriptor.getPrimitiveType().getPrimitiveTypeName();
        checkArgument(

View on GitHub (pinned to 2f3c205e92)