apache/beam · error · IllegalArgumentException

PubSubIO cannot be configured with both a dead letter topic

Error message

PubSubIO cannot be configured with both a dead letter topic and a bad record router

What it means

PubsubIO.Read rejects a configuration that sets both a dead letter topic and a custom BadRecordRouter. These two failure-handling mechanisms are mutually exclusive: the dead letter topic is itself implemented via the bad-record routing path, so an explicit router would conflict with dead-letter delivery. The library throws IllegalArgumentException at pipeline construction time in Read.expand() before any data is processed.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubIO.java:1234

     */
    Read<T> withClock(Clock clock) {
      return toBuilder().setClock(clock).build();
    }

    @Override
    public PCollection<T> expand(PBegin input) {
      if (getTopicProvider() == null && getSubscriptionProvider() == null) {
        throw new IllegalStateException(
            "Need to set either the topic or the subscription for " + "a PubsubIO.Read transform");
      }
      if (getTopicProvider() != null && getSubscriptionProvider() != null) {
        throw new IllegalStateException(
            "Can't set both the topic and the subscription for " + "a PubsubIO.Read transform");
      }

      if (getDeadLetterTopicProvider() != null
          && !(getBadRecordRouter() instanceof ThrowingBadRecordRouter)) {
        throw new IllegalArgumentException(
            "PubSubIO cannot be configured with both a dead letter topic and a bad record router");
      }

      ValueProvider<PubsubTopic> topicProvider = getTopicProvider();
      @Nullable ValueProvider<TopicPath> topicPath =
          topicProvider == null
              ? null
              : NestedValueProvider.of(topicProvider, new TopicPathTranslator());
      ValueProvider<PubsubSubscription> subscriptionProvider = getSubscriptionProvider();
      @Nullable ValueProvider<SubscriptionPath> subscriptionPath =
          subscriptionProvider == null
              ? null
              : NestedValueProvider.of(subscriptionProvider, new SubscriptionPathTranslator());
      PubsubUnboundedSource source =
          new PubsubUnboundedSource(
              getClock(),
              getPubsubClientFactory(),
              null /* always get project from runtime PipelineOptions */,

View on GitHub (pinned to 12126d8942)

Solutions

1. Remove the withBadRecordRouter(...) call and keep withDeadLetterTopic(...) if you want failed records sent to the DLQ.
2. Remove withDeadLetterTopic(...) and keep the custom bad record router if you need custom routing behavior.
3. If you need both behaviors, implement the dead-letter logic inside your custom BadRecordRouter instead.

Example fix

// before
PubsubIO.readMessages()
    .withDeadLetterTopic(topic)
    .withBadRecordRouter(myRouter);
// after
PubsubIO.readMessages()
    .withDeadLetterTopic(topic); // default ThrowingBadRecordRouter

// before expanding the transform
if (io.getDeadLetterTopicProvider() != null && !(io.getBadRecordRouter() instanceof ThrowingBadRecordRouter)) {
  throw new IllegalArgumentException("Choose either deadLetterTopic or a custom BadRecordRouter, not both");
}

When it happens

Trigger: Calling PubsubIO.readMessages()...withDeadLetterTopic(...) while a non-ThrowingBadRecordRouter bad record router is also set (e.g. via withBadRecordRouter(...)), then expanding the transform.

Common situations: Developers copying a pipeline that used a custom bad record router and adding a dead letter topic for durability, or chaining configuration helpers that each set one of the two options.

Related errors

• Need to set either the topic or the subscription for a Pubsu
• Can't set both the topic and the subscription for a PubsubIO
• need to set the topic of a PubsubIO.Write transform if not u
• Could not parse pubsub root url "%s"
• Could not determine port for pubsub root url "%s". You must

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7d8ef56d081b96c1. Report an issue: GitHub.