apache/flink · error · UnsupportedOperationException

Cannot convert Protobuf message with extension field(s)

Error message

Cannot convert Protobuf message with extension field(s)

What it means

For proto2 messages, the writer dispatches by field index into a prebuilt FieldWriter array; extension fields' indices can collide with base-field indices, so writing a message that actually carries a set extension is rejected with UnsupportedOperationException.

Source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/protobuf/PatchedProtoWriteSupport.java:500

                        "Protocol Buffers editions syntax is not supported");
            }

            // proto2 uses empty string or "proto2", proto3 uses "proto3"
            boolean isProto2 = syntax.isEmpty() || "proto2".equals(syntax);

            if (isProto2) {
                // ============================================================================
                // END PATCH
                // ============================================================================
                // Returns changed fields with values. Map is ordered by id.
                Map<FieldDescriptor, Object> changedPbFields = pb.getAllFields();

                for (Map.Entry<FieldDescriptor, Object> entry : changedPbFields.entrySet()) {
                    FieldDescriptor fieldDescriptor = entry.getKey();

                    if (fieldDescriptor.isExtension()) {
                        // Field index of an extension field might overlap with a base field.
                        throw new UnsupportedOperationException(
                                "Cannot convert Protobuf message with extension field(s)");
                    }

                    int fieldIndex = fieldDescriptor.getIndex();
                    fieldWriters[fieldIndex].writeField(entry.getValue());
                }
            } else {
                // proto3
                List<FieldDescriptor> fieldDescriptors = messageDescriptor.getFields();
                for (FieldDescriptor fieldDescriptor : fieldDescriptors) {
                    FieldDescriptor.Type type = fieldDescriptor.getType();

                    // For a field in a oneOf that isn't set don't write anything
                    if (fieldDescriptor.getContainingOneof() != null
                            && !pb.hasField(fieldDescriptor)) {
                        continue;
                    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Do not populate extension fields on messages you write to parquet; clear/strip them before writing
2. Move the extension fields into the base message definition if they must be persisted
3. Convert the data to a proto3 message or a parquet row schema that includes those fields explicitly

Example fix

// before
builder.setExtension(MyExt.extraId, 42);
out.writeRecord(builder.build()); // throws

// after
// fold the field into the message definition and use it directly:
// message User { string name = 1; int64 extra_id = 2; }

boolean hasSetExtensions(MessageOrBuilder pb) {
  for (FieldDescriptor fd : pb.getAllFields().keySet()) { if (fd.isExtension()) return true; }
  return false;
}

try { writer.write(pb); } catch (UnsupportedOperationException e) { if (e.getMessage().contains("extension")) { /* strip extensions, log, or route to alternative sink */ } throw e; }

When it happens

Trigger: Writing a proto2 Message where getAllFields() includes an extension field (i.e., an extension was populated at runtime via extension registry or builder extension setters).

Common situations: Proto2 schemas with extensions (common in older Google/legacy APIs); messages populated by generic tooling that sets extensions; custom options fields materialized as extensions.

Related errors

• Protocol buffer class or descriptor not specified. Please us
• Expected two fields for the map (key/value), but got: {}
• Protocol Buffers editions syntax is not supported
• FieldIndex mismatch name={}: {} != {}
• Unknown type with descriptor "{}" and type "{}."

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/0163b8d2f4d62085. Report an issue: GitHub.