What causes "Invalid PubSub project name: %r." in apache/beam?

Use the canonical lowercase project id (not project number or display name) in the topic path

apache/beam · error · ValueError

Invalid PubSub project name: %r.

Error message

Invalid PubSub project name: %r.

What it means

After the topic path shape matches, parse_topic validates the extracted project segment against PROJECT_ID_REGEXP. A ValueError is raised when the project component of projects/<project>/topics/<topic> is not a valid GCP project id.

Source

Thrown at sdks/python/apache_beam/io/gcp/pubsub.py:494

            self.publish_with_ordering_key,
            label='Publish With Ordering Key').drop_if_none(),
    }


PROJECT_ID_REGEXP = '[a-z][-a-z0-9:.]{4,61}[a-z0-9]'
SUBSCRIPTION_REGEXP = 'projects/([^/]+)/subscriptions/(.+)'
TOPIC_REGEXP = 'projects/([^/]+)/topics/(.+)'


def parse_topic(full_topic: str) -> tuple[str, str]:
  match = re.match(TOPIC_REGEXP, full_topic)
  if not match:
    raise ValueError(
        'PubSub topic must be in the form "projects/<project>/topics'
        '/<topic>" (got %r).' % full_topic)
  project, topic_name = match.group(1), match.group(2)
  if not re.match(PROJECT_ID_REGEXP, project):
    raise ValueError('Invalid PubSub project name: %r.' % project)
  return project, topic_name


def parse_subscription(full_subscription):
  match = re.match(SUBSCRIPTION_REGEXP, full_subscription)
  if not match:
    raise ValueError(
        'PubSub subscription must be in the form "projects/<project>'
        '/subscriptions/<subscription>" (got %r).' % full_subscription)
  project, subscription_name = match.group(1), match.group(2)
  if not re.match(PROJECT_ID_REGEXP, project):
    raise ValueError('Invalid PubSub project name: %r.' % project)
  return project, subscription_name


# TODO(BEAM-27443): Remove (or repurpose as a proper PTransform).
class _PubSubSource(iobase.SourceBase):
  """Source for a Cloud Pub/Sub topic or subscription.

View on GitHub (pinned to 12126d8942)

Solutions

Use the canonical lowercase project id (not project number or display name) in the topic path
Verify with re.match against GCP project id rules before constructing the sink
Fetch the correct project id via `gcloud config get-value project`

Example fix

// before
WriteToPubSub(topic='projects/MyProject123!/topics/t')
// after
WriteToPubSub(topic='projects/myproject123/topics/t')

Defensive patterns

Strategy: validation

Validate before calling

import re
proj = topic.split('/')[1]
assert re.match(r'[a-z][-a-z0-9]{4,28}[a-z0-9]', proj), f'bad project id: {proj!r}'

Try / catch

try:
    beam.io.WriteToPubSub(topic=topic)
except ValueError as e:
    if 'Invalid PubSub project name' in str(e):
        raise SystemExit(f'Fix project in topic path: {e}')

Prevention

Use lowercase project ids, never project numbers or display names
Validate project ids at config load time
Echo the resolved topic path in pipeline logs for review

When it happens

Trigger: Topics strings like 'projects/My_Project!/topics/t' or 'projects//topics/t' (empty project) where the project segment contains characters outside GCP project-id rules (6-30 chars, lowercase letters, digits, hyphens, must start with a letter).

Common situations: Typos in the project id; embedding an uppercase project name or numeric project number where a project id is expected; empty project due to malformed path.

Understand the failure class

Background: "invalid id" errors: invalid identifier format — why libraries reject IDs before lookup, and how to fix them — this error's family across 37 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/393a83b15ceab3c1. Report an issue: GitHub.