{"record":{"id":"17302af213833bde","repo":"apache/beam","slug":"a-pubsub-message-must-not-have-more-than-100-attributes","errorCode":null,"errorMessage":"A pubsub message must not have more than 100 attributes.","messagePattern":"A pubsub message must not have more than 100 attributes\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/pubsub.py","lineNumber":158,"sourceCode":"    at https://cloud.google.com/pubsub/quotas#resource_limits\n\n    Args:\n      proto_msg: str containing a serialized protobuf.\n      for_publish: bool, if True strip out message fields which cannot be\n        published (currently message_id and publish_time) per\n        https://cloud.google.com/pubsub/docs/reference/rpc/google.pubsub.v1#pubsubmessage\n\n    Returns:\n      A str containing a serialized protobuf of type\n      https://cloud.google.com/pubsub/docs/reference/rpc/google.pubsub.v1#google.pubsub.v1.PubsubMessage\n      containing the payload of this object.\n    \"\"\"\n    if len(self.data) > (10_000_000):\n      raise ValueError('A pubsub message data field must not exceed 10MB')\n\n    if self.attributes:\n      if len(self.attributes) > 100:\n        raise ValueError(\n            'A pubsub message must not have more than 100 attributes.')\n      for key, value in self.attributes.items():\n        if len(key) > 256:\n          raise ValueError(\n              'A pubsub message attribute key must not exceed 256 bytes.')\n        if len(value) > 1024:\n          raise ValueError(\n              'A pubsub message attribute value must not exceed 1024 bytes')\n\n    message_id = None\n    publish_time = None\n    if not for_publish:\n      if self.message_id:\n        message_id = self.message_id\n        if self.publish_time:\n          publish_time = self.publish_time\n\n    if len(self.ordering_key) > 1024:","sourceCodeStart":140,"sourceCodeEnd":176,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/pubsub.py#L140-L176","documentation":"PubsubMessage._to_proto_str enforces the Cloud Pub/Sub resource limit of at most 100 attributes per message; the message being serialized carries more, and the API would reject it at publish time, so the error is raised client-side during protobuf conversion.","triggerScenarios":"Calling message_to_proto_str / _to_proto_str on a PubsubMessage whose attributes dict contains more than 100 keys.","commonSituations":"Programmatically generating attributes from a wide record (one attribute per field); merging attribute maps from multiple sources; misusing attributes to carry bulk metadata.","solutions":["Reduce attributes to <= 100 by dropping optional/derived keys.","Batch overflow metadata into the message data payload (e.g. as JSON) instead of attributes.","Compact related attributes into a single key with structured values."],"exampleFix":"// before\nattrs = {f'field_{i}': str(v) for i, v in enumerate(record)}  # may exceed 100\n// after\nessential = {k: record[k] for k in REQUIRED_FIELDS}\nyield PubsubMessage(data=json.dumps(record).encode(), attributes=essential)","handlingStrategy":"validation","validationCode":"if attributes is not None and len(attributes) > 100:\n    raise ValueError(f'too many attributes: {len(attributes)}')","typeGuard":"def fits_pubsub_attribute_count(attrs):\n    return not attrs or len(attrs) <= 100","tryCatchPattern":"try:\n    proto = PubsubMessage(data=data, attributes=attrs)._to_proto_str()\nexcept ValueError as e:\n    if 'more than 100 attributes' in str(e):\n        move_overflow_to_payload(data, attrs)\n    else:\n        raise","preventionTips":["Keep only essential fields in attributes; put bulk metadata in the data payload","Deduplicate/merge attribute maps before constructing messages","Document the 100-attribute cap in your message schema"],"tags":["pubsub","attributes","size-limit","python","apache-beam"],"backgroundTag":"payload-too-large","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}