What causes "deserialize ProtobufNative Schema failed" in apache/pulsar?

Generate the schema info with the client API: Schema.PROTOBUF_NATIVE(MyProto.Msg.class) or ProtobufNativeSchema.of(...), which serializes the FileDescriptorSet correctly.

apache/pulsar · error · InvalidSchemaDataException

deserialize ProtobufNative Schema failed

Error message

deserialize ProtobufNative Schema failed

What it means

ProtobufNativeSchemaDataValidator.validate deserializes the schema definition bytes as a Protobuf FileDescriptorSet via ProtobufNativeSchemaUtils.deserialize; any exception there is rethrown as InvalidSchemaDataException("deserialize ProtobufNative Schema failed"). It means the schema data bytes are not a valid (parseable) FileDescriptorSet.

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/broker/service/schema/validator/ProtobufNativeSchemaDataValidator.java:34

 * specific language governing permissions and limitations
 * under the License.
 */
package org.apache.pulsar.broker.service.schema.validator;

import com.google.protobuf.Descriptors;
import org.apache.pulsar.broker.service.schema.exceptions.InvalidSchemaDataException;
import org.apache.pulsar.client.impl.schema.ProtobufNativeSchemaUtils;
import org.apache.pulsar.common.protocol.schema.SchemaData;

public class ProtobufNativeSchemaDataValidator implements SchemaDataValidator {

    @Override
    public void validate(SchemaData schemaData) throws InvalidSchemaDataException {
        Descriptors.Descriptor descriptor;
        try {
            descriptor = ProtobufNativeSchemaUtils.deserialize(schemaData.getData());
        } catch (Exception e) {
            throw new InvalidSchemaDataException("deserialize ProtobufNative Schema failed", e);
        }
        if (descriptor == null) {
            throw new InvalidSchemaDataException(
                    "protobuf root message descriptor is null,"
                            + " please recheck rootMessageTypeName or rootFileDescriptorName conf. ");
        }
    }

    public static ProtobufNativeSchemaDataValidator of() {
        return INSTANCE;
    }

    private static final ProtobufNativeSchemaDataValidator INSTANCE = new ProtobufNativeSchemaDataValidator();

    private ProtobufNativeSchemaDataValidator() {
    }
}

View on GitHub (pinned to 820761864e)

Solutions

Generate the schema info with the client API: Schema.PROTOBUF_NATIVE(MyProto.Msg.class) or ProtobufNativeSchema.of(...), which serializes the FileDescriptorSet correctly.
If building manually, serialize with ProtobufNativeSchemaUtils.serialize(rootMessageDescriptor) and base64/bytes it as-is.
Verify the bytes round-trip: ProtobufNativeSchemaUtils.deserialize(data) must succeed locally before uploading.
Confirm the client and broker use compatible Pulsar versions for the protobuf-native schema format.

Example fix

// before
SchemaInfo info = SchemaInfoImpl.builder().type(SchemaType.PROTOBUF_NATIVE)
    .data(protoFileText.getBytes(UTF_8)).build(); // .proto text, not a descriptor set
// after
SchemaInfo info = Schema.PROTOBUF_NATIVE(com.example.MyMsg.class)
    .getSchemaInfo(); // serialized FileDescriptorSet

Defensive patterns

Strategy: try-catch

Validate before calling

try {
    com.google.protobuf.Descriptors.FileDescriptor[] fds =
        org.apache.pulsar.client.impl.schema.ProtobufNativeSchemaUtils.deserialize(info.getData());
} catch (Exception e) {
    throw new IllegalArgumentException("Schema data is not a valid FileDescriptorSet", e);
}

Type guard

boolean isProtoNativeSchemaInfo(SchemaInfo info) {
    return info.getType() == SchemaType.PROTOBUF_NATIVE
        && info.getData() != null && info.getData().length > 0;
}

Try / catch

try {
    admin.schemas().createSchema(topic, protoInfo);
} catch (PulsarAdminException e) {
    if (e.getMessage() != null && e.getMessage().contains("deserialize ProtobufNative Schema failed")) {
        // rebuild schema info via Schema.PROTOBUF_NATIVE(msgClass) and retry
    } else {
        throw e;
    }
}

Prevention

Always build protobuf-native schema info with Schema.PROTOBUF_NATIVE(Class) or ProtobufNativeSchema.of(...).
Never paste .proto source text as schema data; it must be the compiled FileDescriptorSet bytes.
Round-trip check the bytes locally (deserialize then compare descriptor names) before uploading.

When it happens

Trigger: Uploading a PROTOBUF_NATIVE schema whose data is not a serialized FileDescriptorSet — e.g. pasting the .proto text as schema data, sending an Avro/JSON blob with type PROTOBUF_NATIVE, or truncated/corrupted descriptor bytes.

Common situations: Hand-crafted admin REST calls where data was base64-encoded incorrectly; scripts that store the .proto source instead of the compiled descriptor; older client versions emitting a legacy descriptor format the broker cannot parse.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/9ae24adaec29e75a. Report an issue: GitHub.