apache/flink · error · IncompatibleSchemaModificationException

FieldIndex mismatch name=

Error message

FieldIndex mismatch name={}: {} != {}

What it means

validatedMapping checks that each protobuf field's declaration index equals its position (field index) in the converted parquet schema. A mismatch means the parquet schema field order does not line up with the descriptor order, which would silently write values to wrong columns; it therefore throws IncompatibleSchemaModificationException.

Solutions

1. Regenerate the parquet schema from the current protobuf descriptor so ordering matches
2. Avoid renaming or reordering proto fields that back parquet columns; add new fields only at the end
3. Keep field names identical (case included) between proto and parquet schema

for (FieldDescriptor fd : descriptor.getFields()) {
  if (parquetSchema.getFieldIndex(fd.getName()) != fd.getIndex()) { throw new IllegalStateException("field order mismatch: " + fd.getFullName()); }
}

When it happens

Trigger: A parquet schema derived from the descriptor whose fields are reordered, renamed (breaking getFieldIndex lookup), or filtered relative to the protobuf descriptor - e.g. custom schema conversion, case-mismatched names, or a descriptor/parquet-schema version skew.

Common situations: Evolving a .proto between writer versions while reusing a stored/derived parquet schema; renaming fields (parquet getFieldIndex is case-sensitive); schema pruning applied to the map type.

Related errors

• Cannot convert Protobuf message with extension field(s)
• Expected two fields for the map (key/value), but got
• Failed to delegate to ProtoSchemaConverter
• Protocol buffer class or descriptor not specified. Please…
• Protocol Buffers editions syntax is not supported

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/17327d4fed06ab32. Report an issue: GitHub.

Appendix: source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/protobuf/PatchedProtoWriteSupport.java:620

    }

    /** validates mapping between protobuffer fields and parquet fields. */
    private void validatedMapping(Descriptor descriptor, GroupType parquetSchema) {
        List<FieldDescriptor> allFields = descriptor.getFields();

        for (FieldDescriptor fieldDescriptor : allFields) {
            String fieldName = fieldDescriptor.getName();
            int fieldIndex = fieldDescriptor.getIndex();
            int parquetIndex = parquetSchema.getFieldIndex(fieldName);
            if (fieldIndex != parquetIndex) {
                String message =
                        "FieldIndex mismatch name="
                                + fieldName
                                + ": "
                                + fieldIndex
                                + " != "
                                + parquetIndex;
                throw new IncompatibleSchemaModificationException(message);
            }
        }
    }

    class StringWriter extends FieldWriter {
        @Override
        final void writeRawValue(Object value) {
            Binary binaryString = Binary.fromString((String) value);
            recordConsumer.addBinary(binaryString);
        }
    }

    class IntWriter extends FieldWriter {
        @Override
        final void writeRawValue(Object value) {
            recordConsumer.addInteger((Integer) value);
        }
    }

View on GitHub (pinned to 2f3c205e92)