{"record":{"id":"f78f7c18db559ce4","repo":"apache/beam","slug":"a-pubsub-message-ordering-key-must-not-exceed-1024-bytes","errorCode":null,"errorMessage":"A pubsub message ordering key must not exceed 1024 bytes.","messagePattern":"A pubsub message ordering key must not exceed 1024 bytes\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/gcp/pubsub.py","lineNumber":177,"sourceCode":"            'A pubsub message must not have more than 100 attributes.')\n      for key, value in self.attributes.items():\n        if len(key) > 256:\n          raise ValueError(\n              'A pubsub message attribute key must not exceed 256 bytes.')\n        if len(value) > 1024:\n          raise ValueError(\n              'A pubsub message attribute value must not exceed 1024 bytes')\n\n    message_id = None\n    publish_time = None\n    if not for_publish:\n      if self.message_id:\n        message_id = self.message_id\n        if self.publish_time:\n          publish_time = self.publish_time\n\n    if len(self.ordering_key) > 1024:\n      raise ValueError(\n          'A pubsub message ordering key must not exceed 1024 bytes.')\n\n    msg = pubsub.types.PubsubMessage(\n        data=self.data,\n        attributes=self.attributes,\n        message_id=message_id,\n        publish_time=publish_time,\n        ordering_key=self.ordering_key)\n    serialized = pubsub.types.PubsubMessage.serialize(msg)\n    if len(serialized) > (10_000_000):\n      raise ValueError(\n          'Serialized pubsub message exceeds the publish request limit of 10MB')\n    return serialized\n\n  @staticmethod\n  def _from_message(msg: Any) -> 'PubsubMessage':\n    \"\"\"Construct from ``google.cloud.pubsub_v1.subscriber.message.Message``.\n","sourceCodeStart":159,"sourceCodeEnd":195,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/gcp/pubsub.py#L159-L195","documentation":"PubsubMessage._to_proto_str enforces the Cloud Pub/Sub limit that the ordering key be at most 1024 bytes; the ordering_key set on this message is longer, so serialization fails client-side rather than at publish.","triggerScenarios":"Serializing a PubsubMessage whose ordering_key (a string set at construction) exceeds 1024 bytes in length.","commonSituations":"Using long composite keys (e.g. concatenated tenant+entity+timestamp strings) as ordering keys; deriving ordering keys from unbounded record fields.","solutions":["Shorten the ordering key to <= 1024 bytes, e.g. by hashing the logical key.","Design ordering keys around a bounded entity id (user/device id) rather than concatenated metadata.","Drop the ordering key entirely if strict ordering is not required (default empty string)."],"exampleFix":"// before\nPubsubMessage(data=b'x', attributes={}, ordering_key='|'.join(record.values()))\n// after\nkey = hashlib.sha1('|'.join(record.values()).encode()).hexdigest()\nyield PubsubMessage(data=b'x', attributes={}, ordering_key=key)","handlingStrategy":"validation","validationCode":"if ordering_key and len(ordering_key) > 1024:\n    raise ValueError(f'ordering_key too long: {len(ordering_key)} bytes')","typeGuard":"def has_valid_ordering_key(msg):\n    return len(getattr(msg, 'ordering_key', '') or '') <= 1024","tryCatchPattern":"try:\n    proto = PubsubMessage(data=data, attributes=attrs, ordering_key=key)._to_proto_str()\nexcept ValueError as e:\n    if 'ordering key must not exceed 1024' in str(e):\n        key = hashlib.sha1(key.encode()).hexdigest()\n    else:\n        raise","preventionTips":["Design ordering keys from bounded entity ids, not concatenated metadata","Hash long logical keys to fixed-length digests","Leave ordering_key empty when strict ordering is not needed"],"tags":["pubsub","ordering-key","size-limit","python","apache-beam"],"backgroundTag":"payload-too-large","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}