apache/flink · error · IncompatibleSchemaModificationException

FieldIndex mismatch name={}: {} != {}

Error message

FieldIndex mismatch name={}: {} != {}

What it means

validatedMapping checks that each protobuf field's declaration index equals its position (field index) in the converted parquet schema. A mismatch means the parquet schema field order does not line up with the descriptor order, which would silently write values to wrong columns; it therefore throws IncompatibleSchemaModificationException.

Source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/protobuf/PatchedProtoWriteSupport.java:620

    }

    /** validates mapping between protobuffer fields and parquet fields. */
    private void validatedMapping(Descriptor descriptor, GroupType parquetSchema) {
        List<FieldDescriptor> allFields = descriptor.getFields();

        for (FieldDescriptor fieldDescriptor : allFields) {
            String fieldName = fieldDescriptor.getName();
            int fieldIndex = fieldDescriptor.getIndex();
            int parquetIndex = parquetSchema.getFieldIndex(fieldName);
            if (fieldIndex != parquetIndex) {
                String message =
                        "FieldIndex mismatch name="
                                + fieldName
                                + ": "
                                + fieldIndex
                                + " != "
                                + parquetIndex;
                throw new IncompatibleSchemaModificationException(message);
            }
        }
    }

    class StringWriter extends FieldWriter {
        @Override
        final void writeRawValue(Object value) {
            Binary binaryString = Binary.fromString((String) value);
            recordConsumer.addBinary(binaryString);
        }
    }

    class IntWriter extends FieldWriter {
        @Override
        final void writeRawValue(Object value) {
            recordConsumer.addInteger((Integer) value);
        }
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Regenerate the parquet schema from the current protobuf descriptor so ordering matches
2. Avoid renaming or reordering proto fields that back parquet columns; add new fields only at the end
3. Keep field names identical (case included) between proto and parquet schema

for (FieldDescriptor fd : descriptor.getFields()) {
  if (parquetSchema.getFieldIndex(fd.getName()) != fd.getIndex()) { throw new IllegalStateException("field order mismatch: " + fd.getFullName()); }
}

When it happens

Trigger: A parquet schema derived from the descriptor whose fields are reordered, renamed (breaking getFieldIndex lookup), or filtered relative to the protobuf descriptor - e.g. custom schema conversion, case-mismatched names, or a descriptor/parquet-schema version skew.

Common situations: Evolving a .proto between writer versions while reusing a stored/derived parquet schema; renaming fields (parquet getFieldIndex is case-sensitive); schema pruning applied to the map type.

Related errors

• Protocol buffer class or descriptor not specified. Please us
• Expected two fields for the map (key/value), but got: {}
• Protocol Buffers editions syntax is not supported
• Cannot convert Protobuf message with extension field(s)
• Unknown type with descriptor "{}" and type "{}."

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/17327d4fed06ab32. Report an issue: GitHub.