apache/beam · error · ValueError

PubSub subscription must be in the form "projects/<project>/

Error message

PubSub subscription must be in the form "projects/<project>/subscriptions/<subscription>" (got %r).

What it means

parse_subscription validates that a Pub/Sub subscription string matches the SUBSCRIPTION_REGEXP form projects/<project>/subscriptions/<subscription>. A ValueError is raised when the string does not match this fully-qualified shape.

Source

Thrown at sdks/python/apache_beam/io/gcp/pubsub.py:501

TOPIC_REGEXP = 'projects/([^/]+)/topics/(.+)'


def parse_topic(full_topic: str) -> tuple[str, str]:
  match = re.match(TOPIC_REGEXP, full_topic)
  if not match:
    raise ValueError(
        'PubSub topic must be in the form "projects/<project>/topics'
        '/<topic>" (got %r).' % full_topic)
  project, topic_name = match.group(1), match.group(2)
  if not re.match(PROJECT_ID_REGEXP, project):
    raise ValueError('Invalid PubSub project name: %r.' % project)
  return project, topic_name


def parse_subscription(full_subscription):
  match = re.match(SUBSCRIPTION_REGEXP, full_subscription)
  if not match:
    raise ValueError(
        'PubSub subscription must be in the form "projects/<project>'
        '/subscriptions/<subscription>" (got %r).' % full_subscription)
  project, subscription_name = match.group(1), match.group(2)
  if not re.match(PROJECT_ID_REGEXP, project):
    raise ValueError('Invalid PubSub project name: %r.' % project)
  return project, subscription_name


# TODO(BEAM-27443): Remove (or repurpose as a proper PTransform).
class _PubSubSource(iobase.SourceBase):
  """Source for a Cloud Pub/Sub topic or subscription.

  This ``NativeSource`` is overridden by a native Pubsub implementation.

  Attributes:
    with_attributes: If False, will fetch just message data. Otherwise,
      fetches ``PubsubMessage`` protobufs.
  """

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass the fully-qualified subscription: 'projects/my-project/subscriptions/my-sub'
2. Build the string from parts: f'projects/{project}/subscriptions/{subscription}'
3. Confirm you passed it to the `subscription=` parameter, not `topic=`

Example fix

// before
ReadFromPubSub(subscription='my-sub')
// after
ReadFromPubSub(subscription='projects/my-gcp-project/subscriptions/my-sub')

import re
assert re.match(r'projects/([^/]+)/subscriptions/(.+)', sub), f'bad subscription: {sub!r}'

try:
    beam.io.ReadFromPubSub(subscription=sub)
except ValueError as e:
    if 'PubSub subscription must be in the form' in str(e):
        sub = f'projects/{PROJECT}/subscriptions/{sub_name}'
    else:
        raise

When it happens

Trigger: Calling ReadFromPubSub with subscription='my-sub' or 'projects/p/subscriptions/' (empty name), or passing a topic path where a subscription is expected.

Common situations: Copying only the subscription id from the console; mixing up topic vs subscription argument; older code using short names from a pre-validation Beam version.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Either data (%r) or attributes (%r) must be set.
• PubSub topic must be in the form "projects/<project>/topics/
• Invalid PubSub project name: %r.
• Either a topic or subscription must be provided.
• Only one of topic or subscription should be provided.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/063989d88d00f5de. Report an issue: GitHub.