apache/beam · error · ValueError

Either a topic or subscription must be provided.

Error message

Either a topic or subscription must be provided.

What it means

_PubSubSource.__init__ requires exactly one of topic or subscription to be specified; a ValueError is raised when neither is provided. This is an upfront argument-validation guard before any pipeline expansion.

Source

Thrown at sdks/python/apache_beam/io/gcp/pubsub.py:538

  def __init__(
      self,
      topic: Optional[str] = None,
      subscription: Optional[str] = None,
      id_label: Optional[str] = None,
      with_attributes: bool = False,
      timestamp_attribute: Optional[str] = None):
    self.coder = coders.BytesCoder()
    self.full_topic = topic
    self.full_subscription = subscription
    self.topic_name = None
    self.subscription_name = None
    self.id_label = id_label
    self.with_attributes = with_attributes
    self.timestamp_attribute = timestamp_attribute

    # Perform some validation on the topic and subscription.
    if not (topic or subscription):
      raise ValueError('Either a topic or subscription must be provided.')
    if topic and subscription:
      raise ValueError('Only one of topic or subscription should be provided.')

    if topic:
      self.project, self.topic_name = parse_topic(topic)
    if subscription:
      self.project, self.subscription_name = parse_subscription(subscription)

  def display_data(self):
    return {
        'id_label': DisplayDataItem(self.id_label,
                                    label='ID Label Attribute').drop_if_none(),
        'topic': DisplayDataItem(self.full_topic,
                                 label='Pubsub Topic').drop_if_none(),
        'subscription': DisplayDataItem(
            self.full_subscription, label='Pubsub Subscription').drop_if_none(),
        'with_attributes': DisplayDataItem(
            self.with_attributes, label='With Attributes').drop_if_none(),

View on GitHub (pinned to 12126d8942)

Solutions

Pass either topic='projects/<project>/topics/<topic>' or subscription='projects/<project>/subscriptions/<sub>'
Check that the config value feeding the argument is not None/empty before building the transform
Default the value at config load time and fail early with a clearer message

Example fix

// before
ReadFromPubSub(topic=os.environ.get('TOPIC'))  # TOPIC unset -> None
// after
topic = os.environ['TOPIC']  # fail fast if missing
assert topic, 'TOPIC must be set'
ReadFromPubSub(topic=topic)

Defensive patterns

Strategy: validation

Validate before calling

assert bool(topic) != bool(subscription) or topic, 'need topic or subscription'
if not (topic or subscription):
    raise ValueError('no Pub/Sub source configured')

Try / catch

try:
    beam.io.ReadFromPubSub(topic=topic, subscription=subscription)
except ValueError as e:
    if 'Either a topic or subscription must be provided' in str(e):
        raise SystemExit('Configure PUBSUB_TOPIC or PUBSUB_SUBSCRIPTION before running')
    raise

Prevention

Fail at config-load time when neither topic nor subscription is present
Use required-config validation before pipeline construction
Never rely on env vars without defaults being set

When it happens

Trigger: Calling ReadFromPubSub() (or _PubSubSource) with no topic and no subscription, e.g. both arguments coming from empty/None config values or unset pipeline parameters.

Common situations: Config-driven pipelines where the pubsub source key is missing from YAML/JSON config; variables resolved to None from environment lookups; refactored code that dropped the default topic.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/41035ff81000506a. Report an issue: GitHub.