apache/pulsar · error · IllegalArgumentException

External schema cannot be used with other Pulsar struct sche

Error message

External schema cannot be used with other Pulsar struct schema types,keySchemaType: ${keySchemaType}, valueSchemaType: ${valueSchemaType}

What it means

KeyValueSchemaImpl's constructor validates the combination of key and value schema types. An EXTERNAL schema (schema fetched from an external registry with unknown Pulsar type) may not be paired with a Pulsar struct type (AVRO/JSON/PROTO) on the other side, because the decoding semantics are incompatible; it throws IllegalArgumentException.

Source

Thrown at pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/KeyValueSchemaImpl.java:121

        this(keySchema, valueSchema, KeyValueEncodingType.INLINE);
    }

    private KeyValueSchemaImpl(Schema<K> keySchema,
                               Schema<V> valueSchema,
                               KeyValueEncodingType keyValueEncodingType) {
        SchemaType keySchemaType = null;
        if (keySchema != null && keySchema.getSchemaInfo() != null) {
            keySchemaType = keySchema.getSchemaInfo().getType();
        }
        SchemaType valueSchemaType = null;
        if (valueSchema != null && valueSchema.getSchemaInfo() != null) {
            valueSchemaType = valueSchema.getSchemaInfo().getType();
        }
        if ((SchemaType.EXTERNAL.equals(keySchemaType)
                && valueSchemaType != null && SchemaType.isStructType(valueSchemaType))
                || (SchemaType.EXTERNAL.equals(valueSchemaType)
                && keySchemaType != null && SchemaType.isStructType(keySchemaType))) {
            throw new IllegalArgumentException("External schema cannot be used with other Pulsar struct schema types,"
                    + "keySchemaType: " + keySchemaType + ", valueSchemaType: " + valueSchemaType);
        }

        this.keySchema = keySchema;
        this.valueSchema = valueSchema;
        this.keyValueEncodingType = keyValueEncodingType;
        this.schemaInfoProvider = new SchemaInfoProvider() {
            @Override
            public CompletableFuture<SchemaInfo> getSchemaByVersion(byte[] schemaVersion) {
                return CompletableFuture.completedFuture(schemaInfo);
            }

            @Override
            public CompletableFuture<SchemaInfo> getLatestSchema() {
                return CompletableFuture.completedFuture(schemaInfo);
            }

            @Override

View on GitHub (pinned to 820761864e)

Solutions

1. Use struct schemas for both key and value (e.g. Schema.AVRO(K.class) and Schema.AVRO(V.class))
2. Or use EXTERNAL consistently for both sides if schemas come from an external registry
3. Inspect getSchemaInfo().getType() of both schemas before composing the KeyValue and pick compatible types

Example fix

// before
Schema<KeyValue<MyKey,MyVal>> s = Schema.KeyValue(Schema.EXTERNAL(SomeSchemaInfo.class), Schema.AVRO(MyVal.class), KeyValueEncodingType.INLINE);
// after
Schema<KeyValue<MyKey,MyVal>> s = Schema.KeyValue(Schema.AVRO(MyKey.class), Schema.AVRO(MyVal.class), KeyValueEncodingType.INLINE);

SchemaType kt = keySchema.getSchemaInfo().getType(); SchemaType vt = valueSchema.getSchemaInfo().getType();
boolean bad = (kt == SchemaType.EXTERNAL && SchemaType.isStructType(vt)) || (vt == SchemaType.EXTERNAL && SchemaType.isStructType(kt));
if (bad) throw new IllegalArgumentException("EXTERNAL schema cannot be paired with struct schema: " + kt + "/" + vt);

boolean compatibleKvPair(Schema<?> k, Schema<?> v) { SchemaType kt = k.getSchemaInfo().getType(); SchemaType vt = v.getSchemaInfo().getType(); return (kt == SchemaType.EXTERNAL) == (vt == SchemaType.EXTERNAL) || (!SchemaType.isStructType(kt) && !SchemaType.isStructType(vt)); }

try { Schema<KeyValue<K,V>> s = Schema.KeyValue(k, v, enc); } catch (IllegalArgumentException e) { if (e.getMessage().contains("External schema cannot be used")) { /* rebuild with matching types */ } else throw e; }

When it happens

Trigger: Constructing Schema.KeyValue(keySchema, valueSchema, encodingType) where one side's SchemaInfo.getType() is EXTERNAL and the other side is a struct type (AVRO/JSON/Protobuf).

Common situations: Mixing an auto-discovered/registry-backed schema with a locally defined POJO schema in a KeyValue; programmatic schema composition where one side's type defaults to EXTERNAL.

Related errors

• AutoConsumeSchema is not supported with schemaId
• The schema is not a KeyValueSchema
• This method cannot be used under this SEPARATED encoding typ
• No key schema info or value schema info : key = ${keySchema.
• Can't get accurate schema information for ${topicName} using

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/5bde5c90c572b374. Report an issue: GitHub.