{"record":{"id":"1da17fff336582a9","repo":"apache/beam","slug":"encountered-pubsub-message-with-ordering-key-but-this-sink","errorCode":null,"errorMessage":"Encountered Pubsub message with ordering key but this sink was not configured to retain ordering keys, so they will be dropped. Please set #withOrderingKeys().","messagePattern":"Encountered Pubsub message with ordering key but this sink was not configured to retain ordering keys, so they will be dropped\\. Please set #withOrderingKeys\\(\\)\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java","lineNumber":212,"sourceCode":"        badRecordRouter.route(\n            o, element, inputCoder, e, \"Failed to determine PubSub topic using topic function\");\n        return;\n      }\n    }\n    String topic = message.getTopic();\n    // topic shouldn't be null, but lineage report is fail-safe\n    if (topic != null && !topic.equals(reportedLineage)) {\n      Lineage.getSinks()\n          .add(\n              \"pubsub\",\n              \"topic\",\n              PubsubClient.topicPathFromPath(topic).getDataCatalogSegments(),\n              null);\n      reportedLineage = topic;\n    }\n    if (!usesOrderingKey && !Strings.isNullOrEmpty(message.getOrderingKey())) {\n      if (!logOrderingKeyUnconfigured) {\n        LOG.warn(\n            \"Encountered Pubsub message with ordering key but this sink was not configured to \"\n                + \"retain ordering keys, so they will be dropped. Please set #withOrderingKeys().\");\n\n        logOrderingKeyUnconfigured = true;\n      }\n      message = message.withOrderingKey(null);\n    }\n    try {\n      validatePubsubMessage(message, maxPublishBatchSize);\n    } catch (SizeLimitExceededException e) {\n      badRecordRouter.route(\n          o,\n          element,\n          inputCoder,\n          new IllegalArgumentException(e),\n          \"PubSub message limit exceeded, see exception for details\");\n      return;\n    }","sourceCodeStart":194,"sourceCodeEnd":230,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java#L194-L230","documentation":"In PreparePubsubWriteDoFn.process, an incoming Pub/Sub message carries an ordering key, but the sink was created without withOrderingKeys(). Since the sink is not configured to publish with ordering keys, the key would cause a publish failure, so it is stripped (set to null) and a one-time warning is logged. Message ordering guarantees are lost for that message.","triggerScenarios":"Reading from a Pub/Sub subscription whose messages have ordering keys (source topic created with message ordering enabled) and writing to another Pub/Sub topic via PubsubIO.write() without .withOrderingKeys(); the first keyed message triggers the warning and key removal.","commonSituations":"Topic-to-topic relay pipelines where the destination sink forgot withOrderingKeys(); source topics migrated to ordered publishing while the downstream Beam pipeline was unchanged; copy-pasted sink builders predating ordering-key support.","solutions":["Add .withOrderingKeys() to the Pub/Sub write transform so ordering keys are retained end-to-end.","Ensure message ordering is enabled on the destination topic and that publishing uses ordering-enabled publishers.","If ordering is irrelevant, ignore the warning; it is logged once per DoFn instance.","Verify source topic configuration: disable ordering keys at the source if downstream does not need them."],"exampleFix":"// before\nPubsubIO.writeMessages().to(topic);\n// after\nPubsubIO.writeMessages().to(topic).withOrderingKeys();","handlingStrategy":"validation","validationCode":"// before running the pipeline, check message ordering is configured end-to-end:\n// source topic has ordering enabled AND sink transform calls withOrderingKeys().","typeGuard":"boolean retainsOrderingKeys(PubsubIO.Write<?> w) { return w != null && w.usesOrderingKeys(); }","tryCatchPattern":null,"preventionTips":["Enable message ordering on both source and destination topics when keys are present.","Always call .withOrderingKeys() on Pub/Sub write transforms in ordered pipelines.","Add a pipeline-level assertion that reads-with-keys map to writes-with-keys.","Review beam warnings once at startup; the warning logs only once per DoFn instance."],"tags":["java","pubsub","ordering-key","configuration","data-loss"],"backgroundTag":"feature-not-enabled","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}