apache/flink · error · InvalidSchemaException

Failed to find related Parquet column descriptor with type {

Error message

Failed to find related Parquet column descriptor with type {}

What it means

InvalidSchemaException from getAllColumnDescriptorByType in ParquetSplitReaderUtil: after scanning all ColumnDescriptors at the given depth, none has a path element equal to the requested type's name, so the logical field cannot be bound to any physical Parquet column. It is the nested-schema analogue of the top-level 'does not exist' error.

Source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/vector/ParquetSplitReaderUtil.java:302

                throw new UnsupportedOperationException("Unsupported type: " + type);
        }
    }

    private static List<ColumnDescriptor> getAllColumnDescriptorByType(
            int depth, Type type, List<ColumnDescriptor> columns) throws ParquetRuntimeException {
        List<ColumnDescriptor> res = new ArrayList<>();
        for (ColumnDescriptor descriptor : columns) {
            if (depth >= descriptor.getPath().length) {
                throw new InvalidSchemaException("Corrupted Parquet schema");
            }
            if (type.getName().equals(descriptor.getPath()[depth])) {
                res.add(descriptor);
            }
        }

        // If doesn't find the type descriptor in corresponding depth, throw exception
        if (res.isEmpty()) {
            throw new InvalidSchemaException(
                    "Failed to find related Parquet column descriptor with type " + type);
        }
        return res;
    }

    public static ColumnReader createColumnReader(
            boolean isUtcTimestamp,
            LogicalType fieldType,
            Type type,
            List<ColumnDescriptor> columnDescriptors,
            PageReadStore pages,
            ParquetField field,
            int depth)
            throws IOException {
        List<ColumnDescriptor> descriptors =
                getAllColumnDescriptorByType(depth, type, columnDescriptors);
        switch (fieldType.getTypeRoot()) {
            case BOOLEAN:

View on GitHub (pinned to 2f3c205e92)

Solutions

Inspect the file's full nested schema with parquet-tools and correct the table DDL (nested field names must match the file)
Regenerate or rewrite files whose nested structure drifted from the table definition
Avoid projecting the mismatched nested subfield if it is not required

Defensive patterns

Strategy: validation

Validate before calling

// check each nested field name resolves at its depth in the file schema
for (int i = 0; i < requested.getFieldCount(); i++) {
    String[] p = requested.getPaths().get(i);
    if (!fileSchema.containsPath(p)) {
        throw new IllegalStateException("Nested column not found in file: " + Arrays.toString(p));
    }
}

Try / catch

catch (InvalidSchemaException e) { if (e.getMessage().contains("Failed to find related Parquet column descriptor")) { /* fix nested field names in DDL */ } else throw e; }

Prevention

Name nested row fields identically in DDL and writer schema
Avoid renaming nested fields during schema evolution without rewriting files

When it happens

Trigger: Resolving a nested field (inside row/array/map) whose name does not match any column path element at that depth - e.g. table declares row<f0 INT, g1 INT> but the file's nested group contains only f0; or case-sensitive mismatch of nested names.

Common situations: Renamed or reordered nested fields after schema evolution; hand-written DDL that guesses nested field names; case differences in nested field names between writer and reader.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/9f971df43e8e817b. Report an issue: GitHub.