apache/pulsar · error · IllegalArgumentException

metadata has a max size of + maxConsumerMetadataSize + byte

Error message

metadata has a max size of + maxConsumerMetadataSize +  bytes

What it means

Metadata.validateMetadata sums the character lengths of all keys and values in a consumer metadata map and throws IllegalArgumentException when the total exceeds maxConsumerMetadataSize. The error message reports the configured maximum size in bytes.

Source

Thrown at pulsar-common/src/main/java/org/apache/pulsar/common/naming/Metadata.java:40

/**
 * Validator for metadata configuration.
 */
public class Metadata {

    private Metadata() {}

    public static void validateMetadata(Map<String, String> metadata,
                                        int maxConsumerMetadataSize) throws IllegalArgumentException {
        if (metadata == null) {
            return;
        }

        int size = 0;
        for (Map.Entry<String, String> e : metadata.entrySet()) {
            size += (e.getKey().length() + e.getValue().length());
            if (size > maxConsumerMetadataSize) {
                throw new IllegalArgumentException(getErrorMessage(maxConsumerMetadataSize));
            }
        }
    }

    private static String getErrorMessage(int maxConsumerMetadataSize) {
        return "metadata has a max size of " + maxConsumerMetadataSize + " bytes";
    }
}

View on GitHub (pinned to 820761864e)

Solutions

1. Reduce the number and size of metadata keys/values attached to the consumer
2. Raise maxConsumerMetadataSize via its configuration if the larger payload is intended
3. Encode or shorten values (e.g. hash long ids instead of embedding them)

Example fix

// before
Map<String, String> meta = Map.of("owner", veryLongOwnershipString); // exceeds limit
// after
Map<String, String> meta = Map.of("ownerHash", sha256Short(veryLongOwnershipString));

int max = 5000; // mirror configured maxConsumerMetadataSize
int size = 0;
for (Map.Entry<String, String> e : metadata.entrySet()) {
    size += e.getKey().length() + e.getValue().length();
}
if (size > max) {
    throw new IllegalArgumentException("Consumer metadata too large: " + size + " > " + max + " chars");
}

try {
    Metadata.validateMetadata(metadata, maxConsumerMetadataSize);
} catch (IllegalArgumentException e) {
    metadata = compactMetadata(metadata); // trim/hash oversized values
}

When it happens

Trigger: Calling Metadata.validateMetadata (directly or via consumer creation APIs) with a properties/metadata map whose combined key+value character lengths exceed the configured limit (char count, not UTF-8 bytes, so multibyte chars can trip it sooner in bytes).

Common situations: Consumers attaching large key/value metadata (tracing ids, ownership tags) to subscriptions; many small entries accumulating past the cap; oversized tenant-supplied labels.

Related errors

• Ranges for KeyShared policy must not be empty.
• Timeout during delete operation
• Timeout during close operation
• Timeout during open-cursor operation
• Timeout during delete-cursors operation

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/5660872b3bff561e. Report an issue: GitHub.