apache/pulsar · error · IncompatibleSchemaException

Incompatible Schema from

Error message

Incompatible Schema from %s + to %s

What it means

When both the existing and the new JSON schemas are in the legacy Jackson format, compatibility is decided by comparing their JSON 'id' fields; if the ids differ the schemas are considered incompatible and the full schema payloads are included in the error message.

Solutions

  1. Ensure all producers for the topic use POJOs that yield the same JsonSchema id (same schema definition).
  2. Re-register the topic schema with the new definition in Avro format (delete schema first if needed).
  3. Migrate off the legacy Jackson format to Avro-format JSON schemas so normal Avro compatibility checks apply.

Example fix

// before: different ids
JsonSchema a = new JsonSchema("com.x.A", "schema1"); // id: schema1
JsonSchema b = new JsonSchema("com.x.B", "schema2"); // id: schema2
// after: aligned id
admin.schemas().createSchema(topic, Schema.JSON(Pojo.class).getSchemaInfo());
Defensive patterns

Strategy: try-catch

Validate before calling

// pre-compare legacy JsonSchema ids client-side
JsonSchema from = MAPPER.readValue(existingBytes, JsonSchema.class);
JsonSchema to = MAPPER.readValue(newBytes, JsonSchema.class);
if (!from.getId().equals(to.getId())) throw new IllegalArgumentException("JsonSchema id mismatch");

Type guard

boolean sameJsonSchemaId(JsonSchema a, JsonSchema b) {
    return a != null && b != null && a.getId() != null && a.getId().equals(b.getId());
}

Try / catch

try {
    admin.schemas().createSchema(topic, schemaInfo);
} catch (PulsarAdminException e) {
    if (e.getMessage().startsWith("Incompatible Schema from")) {
        // ids differ: register new schema version or use a new topic
    }
}

Prevention

When it happens

Trigger: checkCompatible -> isCompatibleJsonSchema with both schemas parseable as legacy Jackson JsonSchema documents whose getId() values are not equal.

Common situations: Two different POJOs (or the same POJO renamed/refactored) serialized with legacy clients uploading schemas to the same topic; teams reusing a topic across unrelated JSON message types.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/f8eab6934ff26634. Report an issue: GitHub.

Appendix: source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/JsonSchemaCompatibilityCheck.java:99

            // When legacy format is disabled, the existing schema must be valid Avro.
            // If it's not, this is a defense-in-depth rejection (PIP-464).
            throw new IncompatibleSchemaException(
                    "Incompatible schema: existing schema is not in valid Avro format for SchemaType.JSON");
        } else {
            // broker has schema format with unknown format
            // maybe corrupted?
            // return true to overwrite
        }
    }

    private static final ObjectReader JSON_SCHEMA_READER =
            ObjectMapperFactory.getMapper().reader().forType(JsonSchema.class);
    private void isCompatibleJsonSchema(SchemaData from, SchemaData to) throws IncompatibleSchemaException {
        try {
            JsonSchema fromSchema = JSON_SCHEMA_READER.readValue(from.getData());
            JsonSchema toSchema = JSON_SCHEMA_READER.readValue(to.getData());
            if (!fromSchema.getId().equals(toSchema.getId())) {
                throw new IncompatibleSchemaException(String.format("Incompatible Schema from %s + to %s",
                        new String(from.getData(), UTF_8), new String(to.getData(), UTF_8)));
            }
        } catch (IOException e) {
            throw new IncompatibleSchemaException(e);
        }
    }

    private boolean isAvroSchema(SchemaData schemaData) {
        try {

            Schema.Parser fromParser = new Schema.Parser(StructSchemaDataValidator.COMPATIBLE_NAME_VALIDATOR);
            fromParser.setValidateDefaults(false);
            Schema fromSchema = fromParser.parse(new String(schemaData.getData(), UTF_8));
            return true;
        } catch (Exception e) {
            return false;
        }
    }

View on GitHub (pinned to 820761864e)