{"record":{"id":"10ee0d149bfad2e4","repo":"apache/beam","slug":"a-pubsub-message-attribute-value-must-not-exceed-1024-bytes","errorCode":null,"errorMessage":"A pubsub message attribute value must not exceed 1024 bytes","messagePattern":"A pubsub message attribute value must not exceed 1024 bytes","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/pubsub.py","lineNumber":165,"sourceCode":"\n    Returns:\n      A str containing a serialized protobuf of type\n      https://cloud.google.com/pubsub/docs/reference/rpc/google.pubsub.v1#google.pubsub.v1.PubsubMessage\n      containing the payload of this object.\n    \"\"\"\n    if len(self.data) > (10_000_000):\n      raise ValueError('A pubsub message data field must not exceed 10MB')\n\n    if self.attributes:\n      if len(self.attributes) > 100:\n        raise ValueError(\n            'A pubsub message must not have more than 100 attributes.')\n      for key, value in self.attributes.items():\n        if len(key) > 256:\n          raise ValueError(\n              'A pubsub message attribute key must not exceed 256 bytes.')\n        if len(value) > 1024:\n          raise ValueError(\n              'A pubsub message attribute value must not exceed 1024 bytes')\n\n    message_id = None\n    publish_time = None\n    if not for_publish:\n      if self.message_id:\n        message_id = self.message_id\n        if self.publish_time:\n          publish_time = self.publish_time\n\n    if len(self.ordering_key) > 1024:\n      raise ValueError(\n          'A pubsub message ordering key must not exceed 1024 bytes.')\n\n    msg = pubsub.types.PubsubMessage(\n        data=self.data,\n        attributes=self.attributes,\n        message_id=message_id,","sourceCodeStart":147,"sourceCodeEnd":183,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/pubsub.py#L147-L183","documentation":"PubsubMessage._to_proto_str enforces the Cloud Pub/Sub limit that attribute values be at most 1024 bytes; one of the values in self.attributes exceeds it, so the message cannot be converted to a publishable protobuf.","triggerScenarios":"Calling message_to_proto_str / _to_proto_str where any value in self.attributes has len(value) > 1024.","commonSituations":"Putting large payloads (JSON blobs, stack traces, serialized objects) into attributes; forwarding values from other systems without size checks.","solutions":["Move large values into the message data payload and keep only small metadata in attributes.","Truncate or hash the value before setting it as an attribute.","Compress/encode the value if it compresses under 1024 bytes, though storing in data is cleaner."],"exampleFix":"// before\nattrs['trace'] = traceback_text  # may exceed 1024 bytes\n// after\nattrs['trace_ref'] = write_to_gcs(traceback_text)","handlingStrategy":"validation","validationCode":"bad = [(k, v) for k, v in (attributes or {}).items() if len(v) > 1024]\nif bad:\n    raise ValueError(f'attribute values exceed 1024 bytes: {[k for k, _ in bad]}')","typeGuard":"def has_valid_attribute_values(attrs):\n    return not attrs or all(len(v) <= 1024 for v in attrs.values())","tryCatchPattern":"try:\n    proto = PubsubMessage(data=data, attributes=attrs)._to_proto_str()\nexcept ValueError as e:\n    if 'attribute value must not exceed 1024' in str(e):\n        log.error('Move oversized attribute value into payload or external store')\n    raise","preventionTips":["Never put traces, JSON blobs, or serialized objects in attributes","Truncate or hash oversized values; keep full data in the payload","Validate attribute sizes in a shared message-building helper"],"tags":["pubsub","attributes","size-limit","python","apache-beam"],"backgroundTag":"payload-too-large","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}