apache/flink · error · FlinkRuntimeException

Can not find column io for parquet reader.

Error message

Can not find column io for parquet reader.

What it means

FlinkRuntimeException from lookupColumnByName in ParquetSplitReaderUtil: it first tries groupColumnIO.getChild(columnName), then falls back to a case-insensitive scan over all children; if neither finds the field, binding the logical schema to the file's ColumnIO tree is impossible. Unlike the top-level checks, this covers nested field lookup (buildFieldsList uses it per row field).

Source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/ParquetSplitReaderUtil.java:702

    /**
     * Parquet's column names are case in sensitive. So when we look up columns we first check for
     * exact match, and if that can not find we look for a case-insensitive match.
     */
    public static ColumnIO lookupColumnByName(GroupColumnIO groupColumnIO, String columnName) {
        ColumnIO columnIO = groupColumnIO.getChild(columnName);

        if (columnIO != null) {
            return columnIO;
        }

        for (int i = 0; i < groupColumnIO.getChildrenCount(); i++) {
            if (groupColumnIO.getChild(i).getName().equalsIgnoreCase(columnName)) {
                return groupColumnIO.getChild(i);
            }
        }

        throw new FlinkRuntimeException("Can not find column io for parquet reader.");
    }

    public static GroupColumnIO getMapKeyValueColumn(GroupColumnIO groupColumnIO) {
        while (groupColumnIO.getChildrenCount() == 1) {
            groupColumnIO = (GroupColumnIO) groupColumnIO.getChild(0);
        }
        return groupColumnIO;
    }

    public static ColumnIO getArrayElementColumn(ColumnIO columnIO) {
        while (columnIO instanceof GroupColumnIO && !columnIO.getType().isRepetition(REPEATED)) {
            columnIO = ((GroupColumnIO) columnIO).getChild(0);
        }

        /* Compatible with array has a standard 3-level structure:
         *  optional group my_list (LIST) {
         *     repeated group element {
         *        required binary str (UTF8);

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Align table (including nested row field names) with the file schema shown by parquet-tools
2. Rewrite or backfill files so the expected nested columns exist
3. Drop the missing nested field from the projection

MessageType fileSchema = ParquetFileReader.readFooter(conf, path).getFileMetaData().getSchema();
ColumnIOProducer producer = new ColumnIOProducer();
MessageColumnIO columnIO = producer.getColumnIO(fileSchema);
for (String name : tableFieldNames) { // includes nested names via buildFieldsList
    try { lookupColumnByName0(columnIO, name); } catch (Exception e) { throw new IllegalStateException("Field not in file ColumnIO: " + name); }
}

catch (FlinkRuntimeException e) { if ("Can not find column io for parquet reader.".equals(e.getMessage())) { /* reconcile nested schema with file */ } else throw e; }

When it happens

Trigger: constructField/buildFieldsList asking for a child column (including nested row fields) whose name matches no child of the group ColumnIO, exactly or case-insensitively.

Common situations: Nested schema drift: table declares subfields the file lacks; renamed nested columns; case-insensitive catalogs over files whose nested names differ in more than case; files behind an external table written by a different schema version.

Related errors

• Failed to find related Parquet column descriptor with type {
• Corrupted Parquet schema
• Parquet with case insensitive mode should have no duplicate
• The quality of field type is incompatible with the request s
• Field types must not be null.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/910b246069fd1e78. Report an issue: GitHub.