{"record":{"id":"37e5e452f37e9277","repo":"apache/pulsar","slug":"cannot-parse-encrypted-message-msgmetadata-on","errorCode":null,"errorMessage":"Cannot parse encrypted message + msgMetadata +  on topic  + topicName","messagePattern":"Cannot parse encrypted message \\+ msgMetadata \\+  on topic  \\+ topicName","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"pulsar-common/src/main/java/org/apache/pulsar/common/api/raw/MessageParser.java","lineNumber":100,"sourceCode":"\n            try {\n                Commands.parseMessageMetadata(payload, msgMetadata);\n            } catch (Throwable t) {\n                log.warn()\n                    .attr(\"topic\", topicName)\n                    .attr(\"ledgerId\", ledgerId)\n                    .attr(\"entryId\", entryId)\n                    .log(\"Failed to deserialize metadata for message - Ignoring\");\n                return;\n            }\n\n            if (msgMetadata.hasMarkerType()) {\n                // Ignore marker messages as they don't contain user data\n                return;\n            }\n\n            if (msgMetadata.getEncryptionKeysCount() > 0) {\n                throw new IOException(\"Cannot parse encrypted message \" + msgMetadata + \" on topic \" + topicName);\n            }\n\n            uncompressedPayload = uncompressPayloadIfNeeded(topicName, msgMetadata, headersAndPayload, ledgerId,\n                    entryId, maxMessageSize);\n\n            if (uncompressedPayload == null) {\n                // Message was discarded on decompression error\n                return;\n            }\n\n            final int numMessages = msgMetadata.getNumMessagesInBatch();\n\n            if (numMessages == 1 && !msgMetadata.hasNumMessagesInBatch()) {\n                processor.process(\n                    RawMessageImpl.get(refCntMsgMetadata, null, uncompressedPayload.retain(), ledgerId, entryId, 0));\n            } else {\n                // handle batch message enqueuing; uncompressed payload has all messages in batch\n                receiveIndividualMessagesFromBatch(","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-common/src/main/java/org/apache/pulsar/common/api/raw/MessageParser.java#L82-L118","documentation":"MessageParser.parseMessage extracts raw message payloads for inspection, but it has no decryption capability. If the message metadata contains encryption keys (the producer sent an encrypted message), parsing would expose only ciphertext, so the parser throws IOException 'Cannot parse encrypted message ... on topic ...' rather than returning meaningless bytes.","triggerScenarios":"Using MessageParser (e.g. via RawReader/RawMessage reading or tooling) on a topic where producers enable end-to-end encryption (producer encryption keys configured), so msgMetadata.getEncryptionKeysCount() > 0.","commonSituations":"Running monitoring/replication/inspection tools (e.g. pulsar client raw readers, kafka-on-pulsar connectors) against topics encrypted with cryptoKeyReader-configured producers; forgetting to disable encryption or supply decryption keys for offline parsing tools.","solutions":["Disable producer-side end-to-end encryption on the topic if raw parsing is required (remove CryptoKeyReader/encryptionKey from the producer config).","If messages must stay encrypted, decrypt them with a real Pulsar client that has the CryptoKeyReader configured, instead of using MessageParser.","Catch the IOException and skip/count encrypted messages if the tool only needs a best-effort parse of unencrypted messages."],"exampleFix":"// before\nMessageParser.parseMessage(topicName, ledgerId, entryId, headersAndPayload, msgMetadata, callback, maxMessageSize);\n// after\nif (msgMetadata.getEncryptionKeysCount() > 0) {\n    log.warn(\"Skipping encrypted message {}:{} on {}\", ledgerId, entryId, topicName);\n    return;\n}\nMessageParser.parseMessage(topicName, ledgerId, entryId, headersAndPayload, msgMetadata, callback, maxMessageSize);","handlingStrategy":"try-catch","validationCode":"if (msgMetadata.getEncryptionKeysCount() > 0) {\n    // encrypted: route to a decrypting consumer or skip\n}","typeGuard":null,"tryCatchPattern":"try {\n    MessageParser.parseMessage(topic, ledgerId, entryId, payload, metadata, cb, maxSize);\n} catch (IOException e) {\n    if (e.getMessage() != null && e.getMessage().startsWith(\"Cannot parse encrypted message\")) {\n        // skip or handle via decrypting client\n    } else {\n        throw e;\n    }\n}","preventionTips":["Check getEncryptionKeysCount() before calling MessageParser.","Only use MessageParser on topics whose producers do not enable end-to-end encryption.","For encrypted topics, use a full client with a CryptoKeyReader instead of raw parsing."],"tags":["encryption","message-parsing","raw-reader","security"],"backgroundTag":"encrypted-message-not-parseable","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}