{"record":{"id":"6586517d1fe2429f","repo":"apache/beam","slug":"pubsub-message-contains-attributes-size-attributes-which","errorCode":null,"errorMessage":"Pubsub message contains {attributes.size()} attributes which exceeds the maximum of {PUBSUB_MESSAGE_MAX_ATTRIBUTES}. See https://cloud.google.com/pubsub/quotas#resource_limits","messagePattern":"Pubsub message contains (.+?) attributes which exceeds the maximum of (.+?)\\. See https://cloud\\.google\\.com/pubsub/quotas#resource_limits","errorType":"validation","errorClass":"SizeLimitExceededException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java","lineNumber":98,"sourceCode":"        throw new SizeLimitExceededException(\n            \"Pubsub message ordering key of length \"\n                + orderingKeySize\n                + \" exceeds maximum of \"\n                + ORDERING_KEY_MAX_BYTE_SIZE\n                + \" bytes. See https://cloud.google.com/pubsub/quotas#resource_limits\");\n      }\n      totalSize += orderingKeySize;\n    }\n\n    final @Nullable Map<String, String> attributes = message.getAttributeMap();\n    if (payloadSize == 0 && (attributes == null || attributes.isEmpty())) {\n      throw new IllegalArgumentException(\n          \"Pubsub message must contain a non-empty payload or at least one attribute.\");\n    }\n\n    if (attributes != null) {\n      if (attributes.size() > PUBSUB_MESSAGE_MAX_ATTRIBUTES) {\n        throw new SizeLimitExceededException(\n            \"Pubsub message contains \"\n                + attributes.size()\n                + \" attributes which exceeds the maximum of \"\n                + PUBSUB_MESSAGE_MAX_ATTRIBUTES\n                + \". See https://cloud.google.com/pubsub/quotas#resource_limits\");\n      }\n\n      // Consider attribute encoding overhead, so it doesn't go over the request limits\n      totalSize += attributes.size() * PUBSUB_MESSAGE_ATTRIBUTE_ENCODE_ADDITIONAL_BYTES;\n\n      for (Map.Entry<String, String> attribute : attributes.entrySet()) {\n        String key = attribute.getKey();\n        int keySize = key.getBytes(StandardCharsets.UTF_8).length;\n        if (keySize > PUBSUB_MESSAGE_ATTRIBUTE_MAX_KEY_BYTES) {\n          throw new SizeLimitExceededException(\n              \"Pubsub message attribute key '\"\n                  + key\n                  + \"' exceeds the maximum of \"","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java#L80-L116","documentation":"Pub/Sub limits a message to at most PUBSUB_MESSAGE_MAX_ATTRIBUTES attribute pairs; validatePubsubMessage throws SizeLimitExceededException when the attribute map exceeds this count. This enforces the documented quota before the publish request is sent.","triggerScenarios":"Publishing a PubsubMessage with more than PUBSUB_MESSAGE_MAX_ATTRIBUTES entries in its attribute map through PreparePubsubWriteDoFn.process — often when entire record fields are dumped into attributes.","commonSituations":"Mapping every column of a wide table row to message attributes; forwarding upstream HTTP headers as attributes; fan-out logic that accumulates attributes per hop.","solutions":["Move bulk metadata into the message payload (e.g. JSON body) and keep only routing attributes","Filter attributes to the required subset before publishing","Aggregate overflow metadata as a single serialized attribute or GCS reference","Validate attribute count at construction time"],"exampleFix":"// before\nPubsubMessage msg = new PubsubMessage(payload, allRecordFieldsAsAttributes); // 200 attrs\n// after\nMap<String, String> attrs = ImmutableMap.of(\"type\", \"record\", \"id\", recordId);\nPubsubMessage msg = new PubsubMessage(toJson(record), attrs);","handlingStrategy":"validation","validationCode":"java\nif (attrs != null && attrs.size() > 100) {\n  throw new IllegalArgumentException(\"Too many attributes: \" + attrs.size());\n}","typeGuard":null,"tryCatchPattern":"java\ntry {\n  validatePubsubMessage(msg, maxBatchSize);\n} catch (SizeLimitExceededException e) {\n  msg = new PubsubMessage(toJson(msg.getAttributeMap()).getBytes(UTF_8),\n      ImmutableMap.of(\"type\", \"overflow\"));\n}","preventionTips":["Keep only routing-relevant attributes; move bulk metadata into the payload","Cap attributes at construction time","Don't forward external headers/record fields 1:1 as attributes","Document the attribute-count quota for pipeline authors"],"tags":["java","pubsub","attributes","quota"],"backgroundTag":"payload-too-large","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-15T02:17:10.978Z"}