{"record":{"id":"910b246069fd1e78","repo":"apache/flink","slug":"can-not-find-column-io-for-parquet-reader","errorCode":null,"errorMessage":"Can not find column io for parquet reader.","messagePattern":"Can not find column io for parquet reader\\.","errorType":"exception","errorClass":"FlinkRuntimeException","httpStatus":null,"severity":"error","filePath":"flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/ParquetSplitReaderUtil.java","lineNumber":702,"sourceCode":"\n    /**\n     * Parquet's column names are case in sensitive. So when we look up columns we first check for\n     * exact match, and if that can not find we look for a case-insensitive match.\n     */\n    public static ColumnIO lookupColumnByName(GroupColumnIO groupColumnIO, String columnName) {\n        ColumnIO columnIO = groupColumnIO.getChild(columnName);\n\n        if (columnIO != null) {\n            return columnIO;\n        }\n\n        for (int i = 0; i < groupColumnIO.getChildrenCount(); i++) {\n            if (groupColumnIO.getChild(i).getName().equalsIgnoreCase(columnName)) {\n                return groupColumnIO.getChild(i);\n            }\n        }\n\n        throw new FlinkRuntimeException(\"Can not find column io for parquet reader.\");\n    }\n\n    public static GroupColumnIO getMapKeyValueColumn(GroupColumnIO groupColumnIO) {\n        while (groupColumnIO.getChildrenCount() == 1) {\n            groupColumnIO = (GroupColumnIO) groupColumnIO.getChild(0);\n        }\n        return groupColumnIO;\n    }\n\n    public static ColumnIO getArrayElementColumn(ColumnIO columnIO) {\n        while (columnIO instanceof GroupColumnIO && !columnIO.getType().isRepetition(REPEATED)) {\n            columnIO = ((GroupColumnIO) columnIO).getChild(0);\n        }\n\n        /* Compatible with array has a standard 3-level structure:\n         *  optional group my_list (LIST) {\n         *     repeated group element {\n         *        required binary str (UTF8);","sourceCodeStart":684,"sourceCodeEnd":720,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/ParquetSplitReaderUtil.java#L684-L720","documentation":"FlinkRuntimeException from lookupColumnByName in ParquetSplitReaderUtil: it first tries groupColumnIO.getChild(columnName), then falls back to a case-insensitive scan over all children; if neither finds the field, binding the logical schema to the file's ColumnIO tree is impossible. Unlike the top-level checks, this covers nested field lookup (buildFieldsList uses it per row field).","triggerScenarios":"constructField/buildFieldsList asking for a child column (including nested row fields) whose name matches no child of the group ColumnIO, exactly or case-insensitively.","commonSituations":"Nested schema drift: table declares subfields the file lacks; renamed nested columns; case-insensitive catalogs over files whose nested names differ in more than case; files behind an external table written by a different schema version.","solutions":["Align table (including nested row field names) with the file schema shown by parquet-tools","Rewrite or backfill files so the expected nested columns exist","Drop the missing nested field from the projection"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"MessageType fileSchema = ParquetFileReader.readFooter(conf, path).getFileMetaData().getSchema();\nColumnIOProducer producer = new ColumnIOProducer();\nMessageColumnIO columnIO = producer.getColumnIO(fileSchema);\nfor (String name : tableFieldNames) { // includes nested names via buildFieldsList\n    try { lookupColumnByName0(columnIO, name); } catch (Exception e) { throw new IllegalStateException(\"Field not in file ColumnIO: \" + name); }\n}","typeGuard":null,"tryCatchPattern":"catch (FlinkRuntimeException e) { if (\"Can not find column io for parquet reader.\".equals(e.getMessage())) { /* reconcile nested schema with file */ } else throw e; }","preventionTips":["Validate nested field coverage of external files before partition registration","Use the same schema tooling (parquet-tools) for DDL authoring"],"tags":["parquet","schema","nested-types","column-not-found","flink"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}