apache/pulsar · error · SchemaSerializationException

payload (${payloadSize} bytes) cannot be decoded with schema

Error message

payload (${payloadSize} bytes) cannot be decoded with schema ${schema}

What it means

When decoding a message payload with the consumer's schema, an ArrayIndexOutOfBoundsException from the underlying decoder usually means the message was produced without proper schema validation and its bytes don't fit the schema. MessageImpl.converts this into SchemaSerializationException with the payload size and schema info so the failure is diagnosable.

Source

Thrown at pulsar-client/src/main/java/org/apache/pulsar/client/impl/MessageImpl.java:525

    }


    private KeyValueSchemaImpl<?, ?> getKeyValueSchema() {
        if (schema instanceof AutoConsumeSchema) {
            return (KeyValueSchemaImpl<?, ?>) ((AutoConsumeSchema) schema).getInternalSchema(getSchemaVersion());
        } else {
            return (KeyValueSchemaImpl<?, ?>) schema;
        }
    }

    private T decode(byte[] schemaVersion) {
        try {
            return decodeBySchema(schemaVersion);
        } catch (ArrayIndexOutOfBoundsException e) {
            // It usually means the message was produced without schema check while the message is not compatible with
            // the current schema. Therefore, convert it to SchemaSerializationException with a better description.
            final int payloadSize = payload.readableBytes();
            throw new SchemaSerializationException("payload (" + payloadSize + " bytes) cannot be decoded with schema "
                    + new String(schema.getSchemaInfo().getSchema(), StandardCharsets.UTF_8));
        }
    }

    private T decodeBySchema(byte[] schemaVersion) {
        T value = poolMessage ? schema.decode(payload.nioBuffer(), schemaVersion) : null;
        if (value != null) {
            return value;
        }

        if (null == schemaVersion) {
            return schema.decode(getByteBuffer());
        } else {
            return schema.decode(getByteBuffer(), schemaVersion);
        }
    }

    private T decodeBySchemaId(byte[] schemaId) {

View on GitHub (pinned to 820761864e)

Solutions

Align the producer's schema with the consumer's schema (re-register compatible schema or fix the producer serialization).
Enable schema validation enforcement on the topic/policies so incompatible messages are rejected at publish time.
Read the message with Schema.BYTES/raw and deserialize manually if you must tolerate mixed payloads; or skip/dead-letter undecodable messages via catch of SchemaSerializationException.

Example fix

// before
Message<MyPojo> msg = consumer.receive();
MyPojo v = msg.getValue(); // throws
// after
try {
  MyPojo v = msg.getValue();
} catch (PulsarClientException.SchemaSerializationException e) {
  log.warn("Undecodable message: {}", e.getMessage()); // dead-letter/skip
}

Defensive patterns

Strategy: try-catch

Validate before calling

// before consuming, ensure topic schema is compatible:
// check topic schema version / compatibility policy via admin API against your Schema

Try / catch

try {
  T value = msg.getValue();
} catch (PulsarClientException.SchemaSerializationException e) {
  log.warn("Undecodable payload: {}", e.getMessage());
  // ack/skip or dead-letter the message
}

Prevention

Enforce schema validation on topics (isAllowAutoUpdateSchema / compatibility checks).
Keep producer and consumer schema definitions versioned together.
Catch SchemaSerializationException in the consumer loop to avoid crash loops on poison messages.

When it happens

Trigger: Calling getValue(), getKeyValue(), or getKeyValueBySchemaVersion() on a message whose payload cannot be decoded by the resolved schema — typically messages written by an old/other producer with a different serialization (e.g. incompatible Avro record, JSON written without schema).

Common situations: Schema compatibility set too loosely (ALLOWED/always) allowing incompatible producers; a topic previously written with a different schema type (e.g. JSON then Avro); messages produced by non-Pulsar tools bypassing schema checks.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/56d53162bebba3b4. Report an issue: GitHub.