apache/beam · error · ValueError

Either data (%r) or attributes (%r) must be set.

Error message

Either data (%r) or attributes (%r) must be set.

What it means

PubsubMessage.__init__ requires at least one of data or non-empty attributes. If data is None and attributes is empty/None, a message would carry no payload or metadata at all, which is invalid for Pub/Sub, so ValueError is raised. Note this raise uses %-style formatting arguments, so the rendered message may appear odd, but the condition is straightforward.

Source

Thrown at sdks/python/apache_beam/io/gcp/pubsub.py:97

      and service generated attributes (such as id_label and
      timestamp_attribute). May be None.
    message_id: (str) ID of the message, assigned by the pubsub service when the
      message is published. Guaranteed to be unique within the topic. Will be
      reset to None if the message is being written to pubsub.
    publish_time: (datetime) Time at which the message was published. Will be
      reset to None if the Message is being written to pubsub.
    ordering_key: (str) If non-empty, identifies related messages for which
      publish order is respected by the PubSub subscription.
  """
  def __init__(
      self,
      data,
      attributes,
      message_id=None,
      publish_time=None,
      ordering_key=""):
    if data is None and not attributes:
      raise ValueError(
          'Either data (%r) or attributes (%r) must be set.', data, attributes)
    self.data = data
    self.attributes = attributes
    self.message_id = message_id
    self.publish_time = publish_time
    self.ordering_key = ordering_key

  def __hash__(self):
    return hash((self.data, frozenset(self.attributes.items())))

  def __eq__(self, other):
    return isinstance(other, PubsubMessage) and (
        self.data == other.data and self.attributes == other.attributes)

  def __repr__(self):
    return 'PubsubMessage(%s, %s)' % (self.data, self.attributes)

  @staticmethod

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure at least one of data or attributes is set before constructing: provide a payload, or pass non-empty attributes.
2. Guard the emitting DoFn: skip records that produced neither payload nor attributes.
3. Use a default payload (e.g. b'' if the API permits) only when truly intended; better to fail upstream with a clearer error.

Example fix

// before
yield PubsubMessage(data=parsed.get('data'), attributes=parsed.get('attrs'))  # both may be None
// after
if parsed.get('data') is not None or parsed.get('attrs'):
  yield PubsubMessage(data=parsed.get('data'), attributes=parsed.get('attrs'))

if data is None and not attributes:
    raise ValueError('PubsubMessage needs data or non-empty attributes')

def is_publishable(data, attributes):
    return data is not None or bool(attributes)

try:
    msg = PubsubMessage(data=data, attributes=attrs)
except ValueError as e:
    if 'Either data' in str(e):
        log.warning('Skipping record with no payload and no attributes')
    else:
        raise

When it happens

Trigger: Creating PubsubMessage(data=None, attributes=None or {}) — e.g. a DoFn emitting a message from a record whose payload failed to parse and whose attribute map defaulted to empty.

Common situations: Downstream of parsing failures where both payload and attributes end up None/empty; copy-pasted message construction without setting fields; mapping code that filters out attributes and yields data=None.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• PubSub topic must be in the form "projects/<project>/topics/
• Invalid PubSub project name: %r.
• PubSub subscription must be in the form "projects/<project>/
• Either a topic or subscription must be provided.
• Only one of topic or subscription should be provided.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8402cedf34f959e5. Report an issue: GitHub.