apache/beam · critical · RuntimeException

Unable to subscribe to

Error message

Unable to subscribe to ${subscriptionPath}: 

What it means

PubsubUnboundedSource.read() wraps GeneralSecurityException or IOException thrown while constructing PubsubReader (opening the Pubsub client and subscribing) into RuntimeException("Unable to subscribe to <subscriptionPath>: ", e). It means the pipeline failed to establish a subscription to the given Pubsub subscription at startup. The chained cause holds the real reason (auth, network, quota, missing subscription).

Solutions

1. Read the caused-by chain to identify the real failure (security vs IO) and fix accordingly.
2. Verify the subscription exists: gcloud pubsub subscriptions describe <subscription>.
3. Check credentials: ensure Application Default Credentials are valid in the runtime environment.
4. If using a custom Pubsub root URL, confirm network/proxy reachability from workers.
5. Recreate the subscription if it was deleted before the pipeline start.

Example fix

// before
options.setGoogleApplicationCredentialsPath("/stale/path/key.json");
// after (verify before submitting)
// gcloud pubsub subscriptions describe projects/p/subscriptions/s
// export GOOGLE_APPLICATION_CREDENTIALS=/valid/path/key.json
pipeline.apply("ReadPubsub", PubsubIO.readMessages().fromSubscription(subscriptionPath));

gcloud pubsub subscriptions describe <path>  # run before submitting the pipeline
gcloud auth application-default print-access-token  # verify credentials

null

try { pipeline.run().waitUntilFinish(); } catch (RuntimeException e) { Throwable c = e.getCause(); if (c instanceof IOException) { /* check network/endpoint */ } else if (c instanceof GeneralSecurityException) { /* fix credentials */ } }

When it happens

Trigger: Starting a streaming pipeline whose PubsubIO.read().subscription(...) path cannot be opened: invalid credentials, unreachable Pubsub endpoint (custom rootUrl/firewall), or the subscription does not exist / was deleted.

Common situations: Missing or expired GOOGLE_APPLICATION_CREDENTIALS in Dataflow/worker env; typo in subscription path or wrong project; subscription auto-created earlier was garbage-collected; proxy or private-network blocking pubsub.googleapis.com.

Related errors

• A schema was provided without a data format (or viceversa)…
• API Key is required for writing events.
• Both PubsubTestClientFactory and Clock need to be specified…
• Can not get unique key from solr
• Can't set both the topic and the subscription for a…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d9dc31e2e7ad5aec. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubUnboundedSource.java:1112

          subscriptionPath == null ? null : subscriptionPath.get();
      if (subscriptionPathVal == null) {
        if (checkpoint == null) {
          // This reader has never been started and there was no call to #split;
          // create a single random subscription, which will be kept in the checkpoint.
          subscription = outer.createRandomSubscription(options);
        } else {
          subscription =
              checkStateNotNull(
                  checkpoint.getSubscription(),
                  "Checkpoint must have a subscription path when subscriptionPath is not set.");
        }
      } else {
        subscription = subscriptionPathVal;
      }
      try {
        reader = new PubsubReader(options.as(PubsubOptions.class), this, subscription);
      } catch (GeneralSecurityException | IOException e) {
        throw new RuntimeException("Unable to subscribe to " + subscriptionPath + ": ", e);
      }
      if (checkpoint != null) {
        // NACK all messages we may have lost.
        try {
          // Will BLOCK until NACKed.
          checkpoint.nackAll(reader);
        } catch (IOException e) {
          LOG.error(
              "Pubsub {} cannot have {} lost messages NACKed, ignoring exception.",
              subscriptionPath,
              checkpoint.notYetReadIds.size(),
              e);
        }
      }
      return reader;
    }

    @Override

View on GitHub (pinned to 12126d8942)