apache/flink · error · RuntimeException

Unknown ColumnIO, %s

Error message

Unknown ColumnIO, %s

What it means

RuntimeException from constructField in ParquetSplitReaderUtil while resolving an ARRAY field's element ColumnIO: the columnIO object is expected to be a GroupColumnIO (repeated group) or PrimitiveColumnIO; anything else - in practice impossible for well-formed files since ColumnIO only has those two concrete classes - trips this guard, so hitting it indicates schema/ColumnIO tree corruption or an API-level invariant break.

Source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/ParquetSplitReaderUtil.java:666

            ColumnIO elementTypeColumnIO;
            if (columnIO instanceof GroupColumnIO) {
                GroupColumnIO groupColumnIO = (GroupColumnIO) columnIO;
                if (!StringUtils.isNullOrWhitespaceOnly(filedName)) {
                    while (!Objects.equals(groupColumnIO.getName(), filedName)) {
                        groupColumnIO = (GroupColumnIO) groupColumnIO.getChild(0);
                    }
                    elementTypeColumnIO = groupColumnIO;
                } else {
                    if (arrayType.getElementType() instanceof RowType) {
                        elementTypeColumnIO = groupColumnIO;
                    } else {
                        elementTypeColumnIO = groupColumnIO.getChild(0);
                    }
                }
            } else if (columnIO instanceof PrimitiveColumnIO) {
                elementTypeColumnIO = columnIO;
            } else {
                throw new RuntimeException(String.format("Unknown ColumnIO, %s", columnIO));
            }

            ParquetField field =
                    constructField(
                            new RowType.RowField("", arrayType.getElementType()),
                            getArrayElementColumn(elementTypeColumnIO));
            if (repetitionLevel == field.getRepetitionLevel()) {
                repetitionLevel = columnIO.getParent().getRepetitionLevel();
            }
            return new ParquetGroupField(
                    type, repetitionLevel, definitionLevel, required, ImmutableList.of(field));
        }

        PrimitiveColumnIO primitiveColumnIO = (PrimitiveColumnIO) columnIO;
        return new ParquetPrimitiveField(
                type, required, primitiveColumnIO.getColumnDescriptor(), primitiveColumnIO.getId());
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Regenerate or re-copy the file and retry; treat hitting this as data corruption
2. Compare with another reader (parquet-tools, Spark) to confirm the file is unreadable everywhere
3. Report upstream with the file schema if a valid file reproduces it

catch (RuntimeException e) { if (e.getMessage() != null && e.getMessage().startsWith("Unknown ColumnIO")) { /* treat file as corrupt: quarantine and regenerate */ } else throw e; }

When it happens

Trigger: constructField on an ArrayType where the resolved element columnIO is neither GroupColumnIO nor PrimitiveColumnIO - only reachable with malformed column IO trees built from inconsistent file metadata.

Common situations: Corrupted Parquet metadata; files written by broken experimental writers; extremely rare - most malformed schemas fail earlier in getAllColumnDescriptorByType.

Related errors

• The quality of field type is incompatible with the request s
• expecting more rows but reached last block. Read {} out of {
• Corrupted Parquet schema
• could not decode the dictionary for {}
• totalValueCount == 0

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/4ea16191bb530eb3. Report an issue: GitHub.