apache/beam · error · IllegalArgumentException

Pubsub topic '%s' does not exist.

Error message

Pubsub topic '%s' does not exist.

What it means

During validation of a PubsubIO.Read transform (when withValidation is enabled), the connector checks that the configured topic actually exists in Cloud Pub/Sub by calling topicPathFromName / topic lookup. If the topic cannot be found, it throws IllegalArgumentException with the topic path in the message. This is a fail-fast check so pipelines do not run and fail at runtime.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubIO.java:1414

      PubsubOptions psOptions = options.as(PubsubOptions.class);

      // Validate the existence of the topic.
      ValueProvider<PubsubTopic> topicProvider = getTopicProvider();
      if (topicProvider != null) {
        PubsubTopic topic = topicProvider.get();
        boolean topicExists = true;
        try (PubsubClient pubsubClient =
            getPubsubClientFactory()
                .newClient(getTimestampAttribute(), getIdAttribute(), psOptions)) {
          topicExists =
              pubsubClient.isTopicExists(
                  PubsubClient.topicPathFromName(topic.project, topic.topic));
        } catch (Exception e) {
          throw new RuntimeException(e);
        }

        if (!topicExists) {
          throw new IllegalArgumentException(
              String.format("Pubsub topic '%s' does not exist.", topic));
        }
      }
    }

    @Override
    public void populateDisplayData(DisplayData.Builder builder) {
      super.populateDisplayData(builder);
      populateCommonDisplayData(
          builder, getTimestampAttribute(), getIdAttribute(), getTopicProvider());
      builder.addIfNotNull(
          DisplayData.item("subscription", getSubscriptionProvider())
              .withLabel("Pubsub Subscription"));
    }
  }

  private static class ParseReadFailuresToBadRecords
      extends DoFn<KV<PubsubMessage, EncodableThrowable>, BadRecord> {

View on GitHub (pinned to 12126d8942)

Solutions

  1. Verify the topic exists: gcloud pubsub topics list --project=YOUR_PROJECT; create it with gcloud pubsub topics create if missing.
  2. Check the project segment of the topic path matches the project where the topic was created.
  3. Fix typos or stale configuration values in the topic string.
  4. If existence checking is unwanted, remove withValidation() so the transform skips this pre-flight check.

Example fix

// before
PubsubIO.readMessages().from("projects/my-proj/topics/myTopc").applyValidation();
// after
gcloud pubsub topics create projects/my-proj/topics/myTopic
PubsubIO.readMessages().from("projects/my-proj/topics/myTopic").applyValidation();
Defensive patterns

Strategy: validation

Validate before calling

boolean exists = false;
try (PubsubClient client = PubsubClient.newGCloudClient(PubsubClient.topicPathFromName(project, topic).toString())) {
  exists = client.topicExists(PubsubClient.topicPathFromName(project, topic));
}
if (!exists) throw new IllegalArgumentException("Topic " + topic + " does not exist");

Try / catch

try {
  pipeline.run();
} catch (IllegalArgumentException e) {
  if (e.getMessage() != null && e.getMessage().contains("does not exist")) {
    // provision topic or fix config, then retry launch
  }
}

Prevention

When it happens

Trigger: Enabling validation on PubsubIO.read...from(topic) where the topic string references a topic that does not exist in the given project (typo, wrong project, or topic deleted before launch).

Common situations: Typos in fully-qualified topic names (projects/p/topic), running the pipeline against a different GCP project than where the topic was created, topics deleted between runs, or environment-specific config pointing at staging resources.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7d2f499e66a51323. Report an issue: GitHub.