{"record":{"id":"d9fe255752435186","repo":"apache/beam","slug":"pubsub-message-attribute-value-for-key-key-starting-with","errorCode":null,"errorMessage":"Pubsub message attribute value for key '{key}' starting with '{value.substring(0, Math.min(256, value.length()))}' exceeds the maximum of {PUBSUB_MESSAGE_ATTRIBUTE_MAX_VALUE_BYTES} bytes. See https://cloud.google.com/pubsub/quotas#resource_limits","messagePattern":"Pubsub message attribute value for key '(.+?)' starting with '(.+?)' exceeds the maximum of (.+?) bytes\\. See https://cloud\\.google\\.com/pubsub/quotas#resource_limits","errorType":"validation","errorClass":"SizeLimitExceededException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java","lineNumber":125,"sourceCode":"      totalSize += attributes.size() * PUBSUB_MESSAGE_ATTRIBUTE_ENCODE_ADDITIONAL_BYTES;\n\n      for (Map.Entry<String, String> attribute : attributes.entrySet()) {\n        String key = attribute.getKey();\n        int keySize = key.getBytes(StandardCharsets.UTF_8).length;\n        if (keySize > PUBSUB_MESSAGE_ATTRIBUTE_MAX_KEY_BYTES) {\n          throw new SizeLimitExceededException(\n              \"Pubsub message attribute key '\"\n                  + key\n                  + \"' exceeds the maximum of \"\n                  + PUBSUB_MESSAGE_ATTRIBUTE_MAX_KEY_BYTES\n                  + \" bytes. See https://cloud.google.com/pubsub/quotas#resource_limits\");\n        }\n        totalSize += keySize;\n\n        String value = attribute.getValue();\n        int valueSize = value.getBytes(StandardCharsets.UTF_8).length;\n        if (valueSize > PUBSUB_MESSAGE_ATTRIBUTE_MAX_VALUE_BYTES) {\n          throw new SizeLimitExceededException(\n              \"Pubsub message attribute value for key '\"\n                  + key\n                  + \"' starting with '\"\n                  + value.substring(0, Math.min(256, value.length()))\n                  + \"' exceeds the maximum of \"\n                  + PUBSUB_MESSAGE_ATTRIBUTE_MAX_VALUE_BYTES\n                  + \" bytes. See https://cloud.google.com/pubsub/quotas#resource_limits\");\n        }\n        totalSize += valueSize;\n      }\n    }\n\n    if (totalSize > maxPublishBatchSize) {\n      throw new SizeLimitExceededException(\n          \"Pubsub message of length \"\n              + totalSize\n              + \" exceeds maximum of \"\n              + maxPublishBatchSize","sourceCodeStart":107,"sourceCodeEnd":143,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java#L107-L143","documentation":"PreparePubsubWriteDoFn validates each Pub/Sub message attribute value before publishing. Google Cloud Pub/Sub limits a single attribute value to 1024 bytes (PUBSUB_MESSAGE_ATTRIBUTE_MAX_VALUE_BYTES); when the UTF-8 encoded value exceeds this, a SizeLimitExceededException is thrown with a 256-char preview of the offending value so the developer can locate it.","triggerScenarios":"process() calls validatePubsubMessage(); the byte length of any attribute value (value.getBytes(StandardCharsets.UTF_8).length) exceeds PUBSUB_MESSAGE_ATTRIBUTE_MAX_VALUE_BYTES during a PubsubIO.write with attributes.","commonSituations":"Putting large payloads (JSON blobs, stack traces, URLs with query strings) into message attributes instead of the message body; multi-byte UTF-8 content pushing a seemingly short string over 1024 bytes.","solutions":["Move the large data into the message payload and keep only small metadata in attributes","Truncate or hash the attribute value before publishing (e.g. SHA-256 digest of a large field)","Split the value across multiple attributes if it must be carried as attributes","Shorten the value by removing redundant data such as query parameters or whitespace"],"exampleFix":"// before\nmessage.putAttributes(\"payload\", bigJsonString);\n// after\nmessage.putAttributes(\"payloadHash\", sha256Hex(bigJsonString));","handlingStrategy":"validation","validationCode":"for (Map.Entry<String,String> e : attributes.entrySet()) {\n  if (e.getValue().getBytes(java.nio.charset.StandardCharsets.UTF_8).length > 1024) {\n    throw new IllegalArgumentException(\"Attribute '\" + e.getKey() + \"' value exceeds 1024 bytes\");\n  }\n}","typeGuard":"boolean validAttrValue(String v) { return v != null && v.getBytes(java.nio.charset.StandardCharsets.UTF_8).length <= 1024; }","tryCatchPattern":"try { pipeline.run(); } catch (SizeLimitExceededException e) { log.error(\"Pub/Sub attribute too large: {}\", e.getMessage()); }","preventionTips":["Keep attributes to small metadata (ids, types, timestamps)","Hash or reference large values instead of embedding them","Remember UTF-8 multi-byte chars count more than one byte","Add a unit test asserting attribute sizes under 1024 bytes"],"tags":["java","pubsub","message-size","attributes","quota"],"backgroundTag":"payload-too-large","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-15T02:17:10.978Z"}