apache/flink · error · InvalidRecordException

Unknown type with descriptor "{}" and type "{}."

Error message

Unknown type with descriptor "{}" and type "{}."

What it means

The FieldWriter factory hit a FieldDescriptor whose Java type has no writer implementation in PatchedProtoWriteSupport (beyond the primitives, groups, enums, strings, bytes it handles). unknownType raises InvalidRecordException naming the descriptor and JavaType.

Source

Thrown at flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/protobuf/PatchedProtoWriteSupport.java:853

    }

    class BytesValueWriter extends FieldWriter {
        @Override
        void writeRawValue(Object value) {
            byte[] byteArray = ((BytesValue) value).getValue().toByteArray();
            Binary binary = Binary.fromConstantByteArray(byteArray);
            recordConsumer.addBinary(binary);
        }
    }

    private FieldWriter unknownType(FieldDescriptor fieldDescriptor) {
        String exceptionMsg =
                "Unknown type with descriptor \""
                        + fieldDescriptor
                        + "\" and type \""
                        + fieldDescriptor.getJavaType()
                        + "\".";
        throw new InvalidRecordException(exceptionMsg);
    }
}

/**
 * Minimal schema converter extracting only needed behavior for the patched support. For the test
 * cases we only rely on primitive field mappings, so we can forward directly to the real converter
 * if present; else implement minimal mapping. To minimize risk, we reflectively invoke the original
 * ProtoSchemaConverter if available.
 */
class PatchedProtoSchemaConverter {
    private final ParquetConfiguration configuration;

    PatchedProtoSchemaConverter(ParquetConfiguration configuration) {
        this.configuration = configuration;
    }

    MessageType convert(Descriptor descriptor) {
        try {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Restrict the proto message to supported field types (varint, floating point, bool, string, bytes, enum, message, group-free maps)
2. Read the exception message: the descriptor name and JavaType identify the offending field - restructure that field (e.g. wrap in a message or convert to string/bytes)
3. If you control the format jar, extend the writer with a custom FieldWriter for that type

Set<FieldDescriptor.JavaType> ok = EnumSet.of(INT, LONG, FLOAT, DOUBLE, BOOLEAN, STRING, BYTE_STRING, ENUM, MESSAGE);
for (FieldDescriptor fd : descriptor.getFields()) { if (!ok.contains(fd.getJavaType())) throw new IllegalArgumentException("unsupported field: " + fd.getFullName()); }

try { writer.write(msg); } catch (InvalidRecordException e) { // parse field name from message; restructure that proto field } 

When it happens

Trigger: A protobuf field type the writer does not implement - in parquet-proto this typically surfaces for unusual JavaType values; most commonly triggered by unsupported combinations or new protobuf features (e.g. editions-era types) reaching createWriter.

Common situations: Exotic proto features or versions of protobuf-java introducing types the patched converter does not cover; custom FieldDescriptors with types outside the supported set.

Related errors

• Expected two fields for the map (key/value), but got: {}
• Unsupported type information '%s' for field '%s'.
• Unsupported type '%s' for field '%s'.
• Please use AvroParquetReaders.forGenericRecord(Class<T>) for
• Protocol buffer class or descriptor not specified. Please us

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/7984f10f3f724cac. Report an issue: GitHub.