apache/flink · critical · RuntimeException

Translator {} cannot translate the given pipeline {}.

Error message

Translator {} cannot translate the given pipeline {}.

What it means

Thrown from AbstractColumnReader.prepareNewPage when a page's data encoding is dictionary-based (usesDictionary() == true) but no dictionary was ever loaded for the column chunk. The dictionary page must precede dictionary-encoded data pages in a column chunk; if the reader never saw one (or it was dropped), decoding is impossible. This is a file-structure or reader-protocol problem, not a transient I/O issue.

Source

Thrown at flink-clients/src/main/java/org/apache/flink/client/FlinkPipelineTranslationUtil.java:93

    /** Extracts the execution plan (as JSON) from the given {@link Pipeline}. */
    public static String translateToJSONExecutionPlan(
            ClassLoader userClassloader, Pipeline pipeline) {
        FlinkPipelineTranslator pipelineTranslator =
                getPipelineTranslator(userClassloader, pipeline);
        return pipelineTranslator.translateToJSONExecutionPlan(pipeline);
    }

    private static FlinkPipelineTranslator getPipelineTranslator(
            ClassLoader userClassloader, Pipeline pipeline) {

        StreamGraphTranslator streamGraphTranslator = new StreamGraphTranslator(userClassloader);

        if (streamGraphTranslator.canTranslate(pipeline)) {
            return streamGraphTranslator;
        }

        throw new RuntimeException(
                "Translator "
                        + streamGraphTranslator
                        + " cannot translate "
                        + "the given pipeline "
                        + pipeline
                        + ".");
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Verify the file with `parquet-tools dump <file>` — check that each dictionary-encoded page is preceded by a DICTIONARY_PAGE for the same column chunk
2. Regenerate or re-obtain the file from the source system; if it is genuinely malformed the writer is at fault
3. If files are produced by your own writer, ensure dictionary pages are flushed before dictionary-encoded data pages and that fallback to PLAIN encoding happens when the dictionary overflows
4. As a workaround, rewrite the file with a canonical writer (parquet-mr/Spark) which will emit valid dictionary/PLAIN pages

// Verify each dictionary-encoded page is preceded by a dictionary page
try (ParquetFileReader r = ParquetFileReader.open(conf, path)) {
    PageReadStore pages;
    while ((pages = r.readNextRowGroup()) != null) {
        // parquet-mr guarantees dictionary page delivery via readDictionaryPage();
        // if it returns null for a chunk whose pages use dictionary encodings, reject the file
    }
}

try {
    reader.readToVector(...);
} catch (IOException e) {
    if (e.getMessage() != null && e.getMessage().contains("dictionary was missing")) {
        // deterministic file defect — quarantine the file, do not retry
        quarantine(file);
    } else { throw e; }
}

When it happens

Trigger: A column chunk whose first dictionary-encoded page has no preceding dictionary page (corrupt or malformed file); readers constructed with a PageReader that skips or fails to return the dictionary page; files written by a buggy writer that omits the dictionary page after exceeding the dictionary size limit but still marks pages as dictionary-encoded.

Common situations: Reading Parquet files from heterogeneous sources (Spark, Impala, in-house writers) with non-standard dictionary fallback behavior; truncated files where the dictionary page bytes were cut; version mismatches after a writer library upgrade changed dictionary page emission.

Related errors

• Error while waiting for job to be initialized
• Missing JobID. Specify a Job ID to trigger a savepoint.
• Triggering a detached savepoint for the job {} failed.
• could not decode the dictionary for {}
• Could not get job jar and dependencies from JAR file: {}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/23f599f0b788beaf. Report an issue: GitHub.