apache/flink · error · UnsupportedOperationException

Unsupported encoding: {}

Error message

Unsupported encoding: {}

What it means

UnsupportedOperationException from readPageV1 in AbstractColumnReader: for a DataPageV1, definition levels must be RLE-encoded whenever the column has maxDefinitionLevel > 0 (nested/nullable columns). The guard rejects any other definition-level encoding (notably legacy BIT_PACKED from very old Parquet 1.x writers) before attempting to decode.

Source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/reader/AbstractColumnReader.java:204

                    readBatchFromDictionaryIds(0, rowId, vector, vector.getDictionaryIds());
                }
                vector.setDictionary(null);
                readBatch(rowId, num, vector);
            }

            valuesRead += num;
            rowId += num;
            readNumber -= num;
        }
    }

    private void readPageV1(DataPageV1 page) throws IOException {
        this.pageValueCount = page.getValueCount();
        ValuesReader rlReader = page.getRlEncoding().getValuesReader(descriptor, REPETITION_LEVEL);

        // Initialize the decoders.
        if (page.getDlEncoding() != Encoding.RLE && descriptor.getMaxDefinitionLevel() != 0) {
            throw new UnsupportedOperationException(
                    "Unsupported encoding: " + page.getDlEncoding());
        }
        int bitWidth = BytesUtils.getWidthFromMaxInt(descriptor.getMaxDefinitionLevel());
        this.runLenDecoder = new RunLengthDecoder(bitWidth);
        try {
            BytesInput bytes = page.getBytes();
            ByteBufferInputStream in = bytes.toInputStream();
            rlReader.initFromPage(pageValueCount, in);
            this.runLenDecoder.initFromStream(pageValueCount, in);
            prepareNewPage(page.getValueEncoding(), in);
        } catch (IOException e) {
            throw new IOException("could not read page " + page + " in col " + descriptor, e);
        }
    }

    private void readPageV2(DataPageV2 page) throws IOException {
        this.pageValueCount = page.getValueCount();

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Rewrite the old files with a modern writer (Spark/parquet-mr rewrite job) so def levels are RLE
2. If the files must stay untouched, read them with a legacy-capable tool and convert to a supported encoding
3. Upgrade Flink only if release notes add legacy encoding support - the standard fix is rewriting the data

Example fix

// e.g. rewrite with Spark:
// spark.read.parquet("old").write.mode("Overwrite").parquet("new");

catch (UnsupportedOperationException e) { if (e.getMessage() != null && e.getMessage().startsWith("Unsupported encoding:")) { /* rewrite legacy BIT_PACKED files with a modern writer */ } else throw e; }

When it happens

Trigger: Reading a DataPageV1 whose page.getDlEncoding() != Encoding.RLE on a column with descriptor.getMaxDefinitionLevel() != 0.

Common situations: Ancient files written by early Parquet writers (pre-RLE BIT_PACKED def levels); files passed through lossy converters; some third-party encoders emitting non-standard V1 pages.

Related errors

• could not decode the dictionary for {}
• Error while waiting for job to be initialized
• Translator {} cannot translate the given pipeline {}.
• Could not get job jar and dependencies from JAR file: {}
• Failed to retrieve job list.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/b39becdc5cadedd3. Report an issue: GitHub.