{"record":{"id":"48f90bbdc85ddd62","repo":"apache/beam","slug":"pubsub-message-attribute-key-key-exceeds-the-maximum-of","errorCode":null,"errorMessage":"Pubsub message attribute key '{key}' exceeds the maximum of {PUBSUB_MESSAGE_ATTRIBUTE_MAX_KEY_BYTES} bytes. See https://cloud.google.com/pubsub/quotas#resource_limits","messagePattern":"Pubsub message attribute key '(.+?)' exceeds the maximum of (.+?) bytes\\. See https://cloud\\.google\\.com/pubsub/quotas#resource_limits","errorType":"validation","errorClass":"SizeLimitExceededException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java","lineNumber":113,"sourceCode":"\n    if (attributes != null) {\n      if (attributes.size() > PUBSUB_MESSAGE_MAX_ATTRIBUTES) {\n        throw new SizeLimitExceededException(\n            \"Pubsub message contains \"\n                + attributes.size()\n                + \" attributes which exceeds the maximum of \"\n                + PUBSUB_MESSAGE_MAX_ATTRIBUTES\n                + \". See https://cloud.google.com/pubsub/quotas#resource_limits\");\n      }\n\n      // Consider attribute encoding overhead, so it doesn't go over the request limits\n      totalSize += attributes.size() * PUBSUB_MESSAGE_ATTRIBUTE_ENCODE_ADDITIONAL_BYTES;\n\n      for (Map.Entry<String, String> attribute : attributes.entrySet()) {\n        String key = attribute.getKey();\n        int keySize = key.getBytes(StandardCharsets.UTF_8).length;\n        if (keySize > PUBSUB_MESSAGE_ATTRIBUTE_MAX_KEY_BYTES) {\n          throw new SizeLimitExceededException(\n              \"Pubsub message attribute key '\"\n                  + key\n                  + \"' exceeds the maximum of \"\n                  + PUBSUB_MESSAGE_ATTRIBUTE_MAX_KEY_BYTES\n                  + \" bytes. See https://cloud.google.com/pubsub/quotas#resource_limits\");\n        }\n        totalSize += keySize;\n\n        String value = attribute.getValue();\n        int valueSize = value.getBytes(StandardCharsets.UTF_8).length;\n        if (valueSize > PUBSUB_MESSAGE_ATTRIBUTE_MAX_VALUE_BYTES) {\n          throw new SizeLimitExceededException(\n              \"Pubsub message attribute value for key '\"\n                  + key\n                  + \"' starting with '\"\n                  + value.substring(0, Math.min(256, value.length()))\n                  + \"' exceeds the maximum of \"\n                  + PUBSUB_MESSAGE_ATTRIBUTE_MAX_VALUE_BYTES","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java#L95-L131","documentation":"Pub/Sub limits each attribute key length; validatePubsubMessage throws SizeLimitExceededException when a single attribute key's UTF-8 byte length exceeds PUBSUB_MESSAGE_ATTRIBUTE_MAX_KEY_BYTES. Enforced client-side before publish since the Pub/Sub API would reject it.","triggerScenarios":"Publishing a PubsubMessage whose attribute map contains a key whose UTF-8 byte length exceeds PUBSUB_MESSAGE_ATTRIBUTE_MAX_KEY_BYTES (256) via PreparePubsubWriteDoFn.process.","commonSituations":"Using long header names, full file paths, or concatenated key strings as attribute keys; non-ASCII keys inflating byte count; forwarding arbitrary external metadata keys unfiltered.","solutions":["Shorten the offending attribute key (use a short canonical name like 'src' instead of the full path)","Sanitize/normalize incoming keys to a max byte length before building the message","Move long key information into the payload or a shorter key's value","Validate each key's UTF-8 length at construction time"],"exampleFix":"// before\nattrs.put(\"/very/long/source/system/path/from/external/integration/...\", value);\n// after\nattrs.put(\"source-path\", value); // key well under PUBSUB_MESSAGE_ATTRIBUTE_MAX_KEY_BYTES","handlingStrategy":"validation","validationCode":"java\nfor (Map.Entry<String, String> a : attrs.entrySet()) {\n  if (a.getKey().getBytes(StandardCharsets.UTF_8).length > 256) {\n    throw new IllegalArgumentException(\"Attribute key too long: \" + a.getKey());\n  }\n}","typeGuard":null,"tryCatchPattern":"java\ntry {\n  validatePubsubMessage(msg, maxBatchSize);\n} catch (SizeLimitExceededException e) {\n  if (e.getMessage().contains(\"attribute key\")) {\n    msg = new PubsubMessage(msg.getPayload(), shortenKeys(msg.getAttributeMap()));\n  } else throw e;\n}","preventionTips":["Use short canonical attribute key names","Sanitize external keys to a max UTF-8 byte length before use","Store long key context in the attribute value or payload","Validate all keys at message construction"],"tags":["java","pubsub","attributes","quota"],"backgroundTag":"payload-too-large","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-15T02:17:10.978Z"}