apache/beam · error · IllegalStateException

Need to set either the topic or the subscription for a Pubsu

Error message

Need to set either the topic or the subscription for a PubsubIO.Read transform

What it means

The Read transform's expand() validates its configuration before applying: a PubsubIO.Read must have exactly one source. If neither topic nor subscription was set, this IllegalStateException is thrown during pipeline expansion.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubIO.java:1224

    /** Enable validation of the PubSub Read. */
    public Read<T> withValidation() {
      return toBuilder().setValidate(true).build();
    }

    @VisibleForTesting
    /**
     * Set's the internal Clock.
     *
     * <p>Only for use by unit tests.
     */
    Read<T> withClock(Clock clock) {
      return toBuilder().setClock(clock).build();
    }

    @Override
    public PCollection<T> expand(PBegin input) {
      if (getTopicProvider() == null && getSubscriptionProvider() == null) {
        throw new IllegalStateException(
            "Need to set either the topic or the subscription for " + "a PubsubIO.Read transform");
      }
      if (getTopicProvider() != null && getSubscriptionProvider() != null) {
        throw new IllegalStateException(
            "Can't set both the topic and the subscription for " + "a PubsubIO.Read transform");
      }

      if (getDeadLetterTopicProvider() != null
          && !(getBadRecordRouter() instanceof ThrowingBadRecordRouter)) {
        throw new IllegalArgumentException(
            "PubSubIO cannot be configured with both a dead letter topic and a bad record router");
      }

      ValueProvider<PubsubTopic> topicProvider = getTopicProvider();
      @Nullable ValueProvider<TopicPath> topicPath =
          topicProvider == null
              ? null
              : NestedValueProvider.of(topicProvider, new TopicPathTranslator());

View on GitHub (pinned to 12126d8942)

Solutions

1. Call .fromTopic("projects/P/topics/T") or .fromSubscription("projects/P/subscriptions/S") on the Read transform before applying.
2. Check that your config/argument actually contains the topic or subscription value and that the setter isn't skipped on some code path.
3. Validate configuration at pipeline start so missing source fails early with a clear message.

Example fix

// before
PubsubIO<String> read = PubsubIO.readStrings();
if (cfg.useTopic) read = read.fromTopic(cfg.topic); // useTopic false -> nothing set
// after
checkArgument(cfg.topic != null || cfg.subscription != null, "topic or subscription required");
PubsubIO<String> read = cfg.subscription == null
    ? PubsubIO.readStrings().fromTopic(cfg.topic)
    : PubsubIO.readStrings().fromSubscription(cfg.subscription);

checkArgument(topic != null || subscription != null,
    "PubsubIO.Read requires topic or subscription");
PubsubIO.Read<String> read = subscription != null
    ? PubsubIO.readStrings().fromSubscription(subscription)
    : PubsubIO.readStrings().fromTopic(topic);

boolean hasSource(String topic, String subscription) {
  return topic != null ^ subscription != null; // exactly one
}

try {
  return pipeline.apply(read);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("Need to set either the topic or the subscription"))
    throw new ConfigException("No Pub/Sub source configured; set topic or subscription", e);
  throw e;
}

When it happens

Trigger: Building PubsubIO.read...() and applying it without ever calling fromTopic(...) or fromSubscription(...) — e.g. conditionally setting the source from config where both branches were skipped.

Common situations: Configuration file missing the topic/subscription key; code path where the setter call was commented out or guarded by a false condition.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Can't set both the topic and the subscription for a PubsubIO
• To read from Pubsub, a subscription name or a topic name mus
• Unrecognized value for stable unique names:
• Secret option string cannot be null
• Secret string must contain a valid type parameter

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2e9d95542a994a50. Report an issue: GitHub.