apache/beam · error · ValueError

No publish time present in message: %s

Error message

No publish time present in message: %s

What it means

When no timestamp_attribute is configured, the DirectRunner derives element timestamps from the Pub/Sub message's server-set publish_time. If the message carries no publish time, the evaluator raises ValueError('No publish time present in message').

Source

Thrown at sdks/python/apache_beam/runners/direct/transform_evaluator.py:734

      parsed_message = PubsubMessage._from_message(message)
      if (timestamp_attribute and
          timestamp_attribute in parsed_message.attributes):
        rfc3339_or_milli = parsed_message.attributes[timestamp_attribute]
        try:
          timestamp = Timestamp(micros=int(rfc3339_or_milli) * 1000)
        except ValueError:
          try:
            timestamp = Timestamp.from_rfc3339(rfc3339_or_milli)
          except ValueError as e:
            raise ValueError('Bad timestamp value: %s' % e)
        if timestamp.precision() > Timestamp.MICROS_PRECISION:
          # Element timestamps are limited to microsecond resolution, so
          # ignore sub-microsecond digits, as the Dataflow service does.
          timestamp = timestamp.to_precision(
              Timestamp.MICROS_PRECISION, allow_lossy_conversion=True)
      else:
        if message.publish_time is None:
          raise ValueError('No publish time present in message: %s' % message)
        try:
          timestamp = Timestamp.from_utc_datetime(message.publish_time)
        except ValueError as e:
          raise ValueError('Bad timestamp value for message %s: %s', message, e)

      return timestamp, parsed_message

    # Because of the AutoAck, we are not able to reread messages if this
    # evaluator fails with an exception before emitting a bundle. However,
    # the DirectRunner currently doesn't retry work items anyway, so the
    # pipeline would enter an inconsistent state on any error.
    sub_client = self._get_subscriber_client(self._applied_ptransform)
    response = sub_client.pull(
        subscription=self._sub_name, max_messages=10, timeout=30)
    results = [_get_element(rm.message) for rm in response.received_messages]
    ack_ids = [rm.ack_id for rm in response.received_messages]
    if ack_ids:
      sub_client.acknowledge(subscription=self._sub_name, ack_ids=ack_ids)

View on GitHub (pinned to 12126d8942)

Solutions

1. Set timestamp_attribute in ReadFromPubSub to a publisher-provided attribute carrying the event time.
2. Ensure messages come from the real Pub/Sub service, which always stamps publish_time.
3. In tests/mocks, populate PubsubMessage(publish_time=...) before feeding the evaluator.
4. Upgrade apache-beam if your client library stopped populating publish_time.

Example fix

// before
msg = PubsubMessage(data, attributes)  # publish_time missing
// after
msg = PubsubMessage(data, attributes)
msg.publish_time = datetime.now(timezone.utc)

if msg.publish_time is None and 'event_time' not in msg.attributes:
    raise ValueError('message lacks publish_time and no timestamp_attribute fallback')

def has_usable_timestamp(msg, ts_attr=None) -> bool:
    return (ts_attr is not None and ts_attr in msg.attributes) or msg.publish_time is not None

try:
    read_pubsub()
except ValueError as e:
    if 'No publish time present' in str(e):
        switch_to_timestamp_attribute_source()

When it happens

Trigger: Reading Pub/Sub messages (via the client-based path) whose PubsubMessage.publish_time is None — typically messages published through custom/partial clients or deserialized payloads lacking the field — without a timestamp_attribute fallback.

Common situations: Using google-cloud-pubsub clients or mocks that don't set publish_time; test harnesses constructing PubsubMessage manually; protocol-buffer messages built before publish, before the server stamps the time.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Bad timestamp value: %s
• Bad timestamp value for message %s: %s
• set_watermark expects a Timestamp as input
• Either data (%r) or attributes (%r) must be set.
• A pubsub message data field must not exceed 10MB

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/db679d5c7a8f1bd3. Report an issue: GitHub.