{"record":{"id":"2075591b8147e767","repo":"apache/beam","slug":"pubsub-message-ordering-key-of-length-orderingkeysize","errorCode":null,"errorMessage":"Pubsub message ordering key of length {orderingKeySize} exceeds maximum of {ORDERING_KEY_MAX_BYTE_SIZE} bytes. See https://cloud.google.com/pubsub/quotas#resource_limits","messagePattern":"Pubsub message ordering key of length (.+?) exceeds maximum of (.+?) bytes\\. See https://cloud\\.google\\.com/pubsub/quotas#resource_limits","errorType":"validation","errorClass":"SizeLimitExceededException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java","lineNumber":80,"sourceCode":"\n  static int validatePubsubMessage(PubsubMessage message, int maxPublishBatchSize)\n      throws SizeLimitExceededException {\n    int payloadSize = message.getPayload().length;\n    if (payloadSize > PUBSUB_MESSAGE_DATA_MAX_BYTES) {\n      throw new SizeLimitExceededException(\n          \"Pubsub message data field of length \"\n              + payloadSize\n              + \" exceeds maximum of \"\n              + PUBSUB_MESSAGE_DATA_MAX_BYTES\n              + \" bytes. See https://cloud.google.com/pubsub/quotas#resource_limits\");\n    }\n    int totalSize = payloadSize;\n\n    @Nullable String orderingKey = message.getOrderingKey();\n    if (orderingKey != null) {\n      int orderingKeySize = orderingKey.getBytes(StandardCharsets.UTF_8).length;\n      if (orderingKeySize > ORDERING_KEY_MAX_BYTE_SIZE) {\n        throw new SizeLimitExceededException(\n            \"Pubsub message ordering key of length \"\n                + orderingKeySize\n                + \" exceeds maximum of \"\n                + ORDERING_KEY_MAX_BYTE_SIZE\n                + \" bytes. See https://cloud.google.com/pubsub/quotas#resource_limits\");\n      }\n      totalSize += orderingKeySize;\n    }\n\n    final @Nullable Map<String, String> attributes = message.getAttributeMap();\n    if (payloadSize == 0 && (attributes == null || attributes.isEmpty())) {\n      throw new IllegalArgumentException(\n          \"Pubsub message must contain a non-empty payload or at least one attribute.\");\n    }\n\n    if (attributes != null) {\n      if (attributes.size() > PUBSUB_MESSAGE_MAX_ATTRIBUTES) {\n        throw new SizeLimitExceededException(","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java#L62-L98","documentation":"validatePubsubMessage checks the Pub/Sub ordering key quota (ORDERING_KEY_MAX_BYTE_SIZE); ordering keys longer than that in UTF-8 bytes throw SizeLimitExceededException. Pub/Sub rejects ordering keys larger than the service limit, so the DoFn fails fast client-side.","triggerScenarios":"Writing a PubsubMessage with a non-null ordering key whose UTF-8 byte length exceeds ORDERING_KEY_MAX_BYTE_SIZE through PubsubIO write with ordering keys enabled.","commonSituations":"Using long entity IDs, full URLs, or concatenated multi-field keys as ordering keys; forgetting that non-ASCII characters inflate UTF-8 byte length.","solutions":["Shorten the ordering key: hash or truncate it to a fixed short value (e.g. SHA-256 prefix) while keeping ordering semantics","Validate key length in producer code before building the message","Restrict keys to ASCII and a fixed max length","Keep high-cardinality info in attributes, not the ordering key"],"exampleFix":"// before\nString key = userId + \":\" + fullUrl;\nmessage = new PubsubMessage(payload, attrs).withOrderingKey(key);\n// after\nString key = Hashing.sha256().hashString(userId, UTF_8).toString().substring(0, 16);\nmessage = new PubsubMessage(payload, attrs).withOrderingKey(key);","handlingStrategy":"validation","validationCode":"java\nif (orderingKey != null && orderingKey.getBytes(StandardCharsets.UTF_8).length > 1024) {\n  throw new IllegalArgumentException(\"Ordering key too long: \" + orderingKey);\n}","typeGuard":null,"tryCatchPattern":"java\ntry {\n  validatePubsubMessage(msg, maxBatchSize);\n} catch (SizeLimitExceededException e) {\n  msg = new PubsubMessage(msg.getPayload(), msg.getAttributeMap())\n      .withOrderingKey(shorten(msg.getOrderingKey()));\n}","preventionTips":["Keep ordering keys short and ASCII","Hash long identifiers into fixed-length keys","Validate key length at message construction","Don't concatenate multiple long fields into the ordering key"],"tags":["java","pubsub","ordering-key","quota"],"backgroundTag":"payload-too-large","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-15T02:17:10.978Z"}