apache/pulsar · error · IncompatibleSchemaException

From schema is not a KEY_VALUE schema.

Error message

From schema is not a KEY_VALUE schema.

What it means

KeyValueSchemaCompatibilityCheck requires the previous ('from') schema to also be of KEY_VALUE type; a non-KEY_VALUE previous schema cannot be compared against the new KEY_VALUE schema, so compatibility checking aborts with this error.

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/KeyValueSchemaCompatibilityCheck.java:81

    @Override
    public void checkCompatible(Iterable<SchemaData> from, SchemaData to, SchemaCompatibilityStrategy strategy)
            throws IncompatibleSchemaException {
        if (strategy == SchemaCompatibilityStrategy.ALWAYS_COMPATIBLE) {
            return;
        }
        if (to.getType() != SchemaType.KEY_VALUE) {
            throw new IncompatibleSchemaException("To schema is not a KEY_VALUE schema.");
        }
        LinkedList<SchemaData> fromKeyList = new LinkedList<>();
        LinkedList<SchemaData> fromValueList = new LinkedList<>();
        KeyValue<SchemaData, SchemaData> fromKeyValue;
        KeyValue<SchemaData, SchemaData> toKeyValue = decodeKeyValueSchemaData(to);
        SchemaType toKeyType = toKeyValue.getKey().getType();
        SchemaType toValueType = toKeyValue.getValue().getType();

        for (SchemaData schemaData : from) {
            if (schemaData.getType() != SchemaType.KEY_VALUE) {
                throw new IncompatibleSchemaException("From schema is not a KEY_VALUE schema.");
            }
            fromKeyValue = decodeKeyValueSchemaData(schemaData);
            if (fromKeyValue.getKey().getType() != toKeyType || fromKeyValue.getValue().getType() != toValueType) {
                throw new IncompatibleSchemaException(
                        String.format("Key schemas or Value schemas are different schema type, "
                                        + "from key schema type is %s and to key schema is %s,"
                                        + " from value schema is %s and to value schema is %s",
                                fromKeyValue.getKey().getType(),
                                toKeyType,
                                fromKeyValue.getValue().getType(),
                                toValueType));
            }
            fromKeyList.addFirst(fromKeyValue.getKey());
            fromValueList.addFirst(fromKeyValue.getValue());
        }
        SchemaCompatibilityCheck keyCheck = checkers.getOrDefault(toKeyType, SchemaCompatibilityCheck.DEFAULT);
        SchemaCompatibilityCheck valueCheck = checkers.getOrDefault(toValueType, SchemaCompatibilityCheck.DEFAULT);
        keyCheck.checkCompatible(fromKeyList, toKeyValue.getKey(), strategy);

View on GitHub (pinned to 820761864e)

Solutions

1. Provide a KEY_VALUE schema on the producer side
2. Check that schema type negotiation did not degrade to a primitive type

Example fix

// before: mixed history
admin.schemas().createSchema(topic, avroSchemaInfo); // creates non-KV entry
admin.schemas().createSchema(topic, kvSchemaInfo);   // now fails
// after: consistent KV history
deleteSchema(topic); createSchema(topic, kvSchemaInfo);

List<VersionedSchema> history = admin.schemas().getAllSchemas(topic);
boolean allKv = history.stream().allMatch(v -> v.getSchema().getType() == SchemaType.KEY_VALUE);
if (!allKv) throw new IllegalStateException("Schema history contains non-KV entries; clean history first");

boolean historyAllKeyValue(List<VersionedSchema> history) {
    return history.stream().allMatch(v -> v.getSchema().getType() == SchemaType.KEY_VALUE);
}

try {
    admin.schemas().createSchema(topic, kvSchemaInfo);
} catch (PulsarAdminException e) {
    if (e.getMessage().contains("From schema is not a KEY_VALUE schema")) {
        // clean schema history: delete schema/topic and recreate with KV only
    }
}

When it happens

Trigger: checkCompatible(Iterable<SchemaData>, SchemaData, strategy) iterating the existing schema versions where one stored entry has type != KEY_VALUE while the new 'to' schema is KEY_VALUE.

Common situations: Schema history polluted by an earlier non-KV upload (e.g. someone temporarily uploaded a plain AVRO schema before KV enforcement); migration scripts that mixed schema types on one topic.

Related errors

• To schema is not a KEY_VALUE schema.
• Key schemas or Value schemas are different schema type, from
• RestException(conversionError)
• External schema is not compatible with the other schema type
• payload (${payloadSize} bytes) cannot be decoded with schema

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/3f2cadf2751e10a2. Report an issue: GitHub.