{"record":{"id":"23f599f0b788beaf","repo":"apache/flink","slug":"translator-cannot-translate-the-given-pipeline","errorCode":null,"errorMessage":"Translator {} cannot translate the given pipeline {}.","messagePattern":"Translator (.+?) cannot translate the given pipeline (.+?)\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"flink-clients/src/main/java/org/apache/flink/client/FlinkPipelineTranslationUtil.java","lineNumber":93,"sourceCode":"\n    /** Extracts the execution plan (as JSON) from the given {@link Pipeline}. */\n    public static String translateToJSONExecutionPlan(\n            ClassLoader userClassloader, Pipeline pipeline) {\n        FlinkPipelineTranslator pipelineTranslator =\n                getPipelineTranslator(userClassloader, pipeline);\n        return pipelineTranslator.translateToJSONExecutionPlan(pipeline);\n    }\n\n    private static FlinkPipelineTranslator getPipelineTranslator(\n            ClassLoader userClassloader, Pipeline pipeline) {\n\n        StreamGraphTranslator streamGraphTranslator = new StreamGraphTranslator(userClassloader);\n\n        if (streamGraphTranslator.canTranslate(pipeline)) {\n            return streamGraphTranslator;\n        }\n\n        throw new RuntimeException(\n                \"Translator \"\n                        + streamGraphTranslator\n                        + \" cannot translate \"\n                        + \"the given pipeline \"\n                        + pipeline\n                        + \".\");\n    }\n}\n","sourceCodeStart":75,"sourceCodeEnd":102,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-clients/src/main/java/org/apache/flink/client/FlinkPipelineTranslationUtil.java#L75-L102","documentation":"Thrown from AbstractColumnReader.prepareNewPage when a page's data encoding is dictionary-based (usesDictionary() == true) but no dictionary was ever loaded for the column chunk. The dictionary page must precede dictionary-encoded data pages in a column chunk; if the reader never saw one (or it was dropped), decoding is impossible. This is a file-structure or reader-protocol problem, not a transient I/O issue.","triggerScenarios":"A column chunk whose first dictionary-encoded page has no preceding dictionary page (corrupt or malformed file); readers constructed with a PageReader that skips or fails to return the dictionary page; files written by a buggy writer that omits the dictionary page after exceeding the dictionary size limit but still marks pages as dictionary-encoded.","commonSituations":"Reading Parquet files from heterogeneous sources (Spark, Impala, in-house writers) with non-standard dictionary fallback behavior; truncated files where the dictionary page bytes were cut; version mismatches after a writer library upgrade changed dictionary page emission.","solutions":["Verify the file with `parquet-tools dump <file>` — check that each dictionary-encoded page is preceded by a DICTIONARY_PAGE for the same column chunk","Regenerate or re-obtain the file from the source system; if it is genuinely malformed the writer is at fault","If files are produced by your own writer, ensure dictionary pages are flushed before dictionary-encoded data pages and that fallback to PLAIN encoding happens when the dictionary overflows","As a workaround, rewrite the file with a canonical writer (parquet-mr/Spark) which will emit valid dictionary/PLAIN pages"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Verify each dictionary-encoded page is preceded by a dictionary page\ntry (ParquetFileReader r = ParquetFileReader.open(conf, path)) {\n    PageReadStore pages;\n    while ((pages = r.readNextRowGroup()) != null) {\n        // parquet-mr guarantees dictionary page delivery via readDictionaryPage();\n        // if it returns null for a chunk whose pages use dictionary encodings, reject the file\n    }\n}","typeGuard":null,"tryCatchPattern":"try {\n    reader.readToVector(...);\n} catch (IOException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"dictionary was missing\")) {\n        // deterministic file defect — quarantine the file, do not retry\n        quarantine(file);\n    } else { throw e; }\n}","preventionTips":["Only accept files from writers that emit valid dictionary page sequences (parquet-mr, Spark, parquet-cpp defaults)","Quarantine + validate files at the pipeline boundary instead of retrying deterministic decode failures","Include file identity (path, ETag) in error logs to correlate with the offending producer"],"tags":["parquet","encoding","dictionary","data-corruption"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}