Encountered Pubsub message with ordering key but this sink…

Error message

Encountered Pubsub message with ordering key but this sink was not configured to retain ordering keys, so they will be dropped. Please set #withOrderingKeys().

What it means

In PreparePubsubWriteDoFn.process, an incoming Pub/Sub message carries an ordering key, but the sink was created without withOrderingKeys(). Since the sink is not configured to publish with ordering keys, the key would cause a publish failure, so it is stripped (set to null) and a one-time warning is logged. Message ordering guarantees are lost for that message.

Solutions

Add .withOrderingKeys() to the Pub/Sub write transform so ordering keys are retained end-to-end.
Ensure message ordering is enabled on the destination topic and that publishing uses ordering-enabled publishers.
If ordering is irrelevant, ignore the warning; it is logged once per DoFn instance.
Verify source topic configuration: disable ordering keys at the source if downstream does not need them.

Example fix

// before
PubsubIO.writeMessages().to(topic);
// after
PubsubIO.writeMessages().to(topic).withOrderingKeys();

Defensive patterns

Strategy: validation

Validate before calling

// before running the pipeline, check message ordering is configured end-to-end:
// source topic has ordering enabled AND sink transform calls withOrderingKeys().

Type guard

boolean retainsOrderingKeys(PubsubIO.Write<?> w) { return w != null && w.usesOrderingKeys(); }

Prevention

Enable message ordering on both source and destination topics when keys are present.
Always call .withOrderingKeys() on Pub/Sub write transforms in ordered pipelines.
Add a pipeline-level assertion that reads-with-keys map to writes-with-keys.
Review beam warnings once at startup; the warning logs only once per DoFn instance.

When it happens

Trigger: Reading from a Pub/Sub subscription whose messages have ordering keys (source topic created with message ordering enabled) and writing to another Pub/Sub topic via PubsubIO.write() without .withOrderingKeys(); the first keyed message triggers the warning and key removal.

Common situations: Topic-to-topic relay pipelines where the destination sink forgot withOrderingKeys(); source topics migrated to ordered publishing while the downstream Beam pipeline was unchanged; copy-pasted sink builders predating ordering-key support.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1da17fff336582a9. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java:212

        badRecordRouter.route(
            o, element, inputCoder, e, "Failed to determine PubSub topic using topic function");
        return;
      }
    }
    String topic = message.getTopic();
    // topic shouldn't be null, but lineage report is fail-safe
    if (topic != null && !topic.equals(reportedLineage)) {
      Lineage.getSinks()
          .add(
              "pubsub",
              "topic",
              PubsubClient.topicPathFromPath(topic).getDataCatalogSegments(),
              null);
      reportedLineage = topic;
    }
    if (!usesOrderingKey && !Strings.isNullOrEmpty(message.getOrderingKey())) {
      if (!logOrderingKeyUnconfigured) {
        LOG.warn(
            "Encountered Pubsub message with ordering key but this sink was not configured to "
                + "retain ordering keys, so they will be dropped. Please set #withOrderingKeys().");

        logOrderingKeyUnconfigured = true;
      }
      message = message.withOrderingKey(null);
    }
    try {
      validatePubsubMessage(message, maxPublishBatchSize);
    } catch (SizeLimitExceededException e) {
      badRecordRouter.route(
          o,
          element,
          inputCoder,
          new IllegalArgumentException(e),
          "PubSub message limit exceeded, see exception for details");
      return;
    }

View on GitHub (pinned to 12126d8942)