apache/pulsar · error · IOException

Cannot parse encrypted message + msgMetadata + on topic +

Error message

Cannot parse encrypted message + msgMetadata +  on topic  + topicName

What it means

MessageParser.parseMessage extracts raw message payloads for inspection, but it has no decryption capability. If the message metadata contains encryption keys (the producer sent an encrypted message), parsing would expose only ciphertext, so the parser throws IOException 'Cannot parse encrypted message ... on topic ...' rather than returning meaningless bytes.

Source

Thrown at pulsar-common/src/main/java/org/apache/pulsar/common/api/raw/MessageParser.java:100

            try {
                Commands.parseMessageMetadata(payload, msgMetadata);
            } catch (Throwable t) {
                log.warn()
                    .attr("topic", topicName)
                    .attr("ledgerId", ledgerId)
                    .attr("entryId", entryId)
                    .log("Failed to deserialize metadata for message - Ignoring");
                return;
            }

            if (msgMetadata.hasMarkerType()) {
                // Ignore marker messages as they don't contain user data
                return;
            }

            if (msgMetadata.getEncryptionKeysCount() > 0) {
                throw new IOException("Cannot parse encrypted message " + msgMetadata + " on topic " + topicName);
            }

            uncompressedPayload = uncompressPayloadIfNeeded(topicName, msgMetadata, headersAndPayload, ledgerId,
                    entryId, maxMessageSize);

            if (uncompressedPayload == null) {
                // Message was discarded on decompression error
                return;
            }

            final int numMessages = msgMetadata.getNumMessagesInBatch();

            if (numMessages == 1 && !msgMetadata.hasNumMessagesInBatch()) {
                processor.process(
                    RawMessageImpl.get(refCntMsgMetadata, null, uncompressedPayload.retain(), ledgerId, entryId, 0));
            } else {
                // handle batch message enqueuing; uncompressed payload has all messages in batch
                receiveIndividualMessagesFromBatch(

View on GitHub (pinned to 820761864e)

Solutions

Disable producer-side end-to-end encryption on the topic if raw parsing is required (remove CryptoKeyReader/encryptionKey from the producer config).
If messages must stay encrypted, decrypt them with a real Pulsar client that has the CryptoKeyReader configured, instead of using MessageParser.
Catch the IOException and skip/count encrypted messages if the tool only needs a best-effort parse of unencrypted messages.

Example fix

// before
MessageParser.parseMessage(topicName, ledgerId, entryId, headersAndPayload, msgMetadata, callback, maxMessageSize);
// after
if (msgMetadata.getEncryptionKeysCount() > 0) {
    log.warn("Skipping encrypted message {}:{} on {}", ledgerId, entryId, topicName);
    return;
}
MessageParser.parseMessage(topicName, ledgerId, entryId, headersAndPayload, msgMetadata, callback, maxMessageSize);

Defensive patterns

Strategy: try-catch

Validate before calling

if (msgMetadata.getEncryptionKeysCount() > 0) {
    // encrypted: route to a decrypting consumer or skip
}

Try / catch

try {
    MessageParser.parseMessage(topic, ledgerId, entryId, payload, metadata, cb, maxSize);
} catch (IOException e) {
    if (e.getMessage() != null && e.getMessage().startsWith("Cannot parse encrypted message")) {
        // skip or handle via decrypting client
    } else {
        throw e;
    }
}

Prevention

Check getEncryptionKeysCount() before calling MessageParser.
Only use MessageParser on topics whose producers do not enable end-to-end encryption.
For encrypted topics, use a full client with a CryptoKeyReader instead of raw parsing.

When it happens

Trigger: Using MessageParser (e.g. via RawReader/RawMessage reading or tooling) on a topic where producers enable end-to-end encryption (producer encryption keys configured), so msgMetadata.getEncryptionKeysCount() > 0.

Common situations: Running monitoring/replication/inspection tools (e.g. pulsar client raw readers, kafka-on-pulsar connectors) against topics encrypted with cryptoKeyReader-configured producers; forgetting to disable encryption or supply decryption keys for offline parsing tools.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/37e5e452f37e9277. Report an issue: GitHub.