apache/beam · error · IllegalStateException

Received message missing publishTime

Error message

Received message missing publishTime

What it means

PubsubJsonClient.pull() converts a raw Pub/Sub response message into an IncomingMessage. When no custom timestamp attribute is configured, the client relies on the server-provided publishTime; if the API response omits it, IllegalStateException "Received message missing publishTime" is thrown. This indicates a malformed or incomplete response from the Pub/Sub service.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java:205

      PubsubMessage pubsubMessage = message.getMessage();
      if (pubsubMessage == null) {
        continue;
      }
      Map<String, String> attributes = pubsubMessage.getAttributes();
      if (attributes == null) {
        attributes = new HashMap<>();
      }

      // Payload.
      String dataStr = pubsubMessage.getData();
      byte[] elementBytes = (dataStr == null) ? new byte[0] : pubsubMessage.decodeData();

      // Timestamp.
      long timestampMsSinceEpoch;
      if (timestampAttribute == null || timestampAttribute.isEmpty()) {
        String publishTime = pubsubMessage.getPublishTime();
        if (publishTime == null) {
          throw new IllegalStateException("Received message missing publishTime");
        }
        timestampMsSinceEpoch = parseTimestampAsMsSinceEpoch(publishTime);
      } else {
        timestampMsSinceEpoch = extractTimestampAttribute(timestampAttribute, attributes);
      }

      // Ack id.
      String ackId = message.getAckId();
      if (ackId == null || ackId.isEmpty()) {
        throw new IllegalStateException("Received message missing ackId");
      }

      // Record id, if any.
      @Nullable String recordId = null;
      if (idAttribute != null) {
        recordId = attributes.get(idAttribute);
      }
      if (recordId == null || recordId.isEmpty()) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Check whether a Pub/Sub emulator/proxy is in play and whether it populates publishTime in pulled messages.
2. Configure a custom timestamp attribute (withTimestampAttribute) so the client reads the timestamp from message attributes instead of the server field.
3. Upgrade the Google API client / Beam version in case of known response-shape incompatibilities.
4. Log the raw response to confirm which messages lack publishTime.

Example fix

// before
PubsubIO.readMessages().from(topic); // relies on server publishTime
// after
PubsubIO.readMessages().from(topic).withTimestampAttribute("myTimestampAttr");

// guard against backends that omit publishTime by using an attribute
if (pubsubMessage.getPublishTime() == null && (timestampAttribute == null || timestampAttribute.isEmpty())) {
  throw new IllegalStateException("Backend does not supply publishTime; configure withTimestampAttribute");
}

try {
  incoming = client.pull(request);
} catch (IllegalStateException e) {
  if (e.getMessage() != null && e.getMessage().contains("missing publishTime")) {
    // fall back to processing-time timestamp or switch to timestamp attribute
  }
}

When it happens

Trigger: Calling pull() on a PubsubJsonClient (no timestampAttribute configured) where a returned message has pubsubMessage.getPublishTime() == null — e.g. an unexpected/empty server response shape, or a stubbed/mocked Pub/Sub backend.

Common situations: Pointing the client at an emulator or proxy that doesn't set publishTime, API version drift changing response fields, or parsing responses from a service that mimics but does not fully implement the Pub/Sub API.

Related errors

• Received message missing ackId
• Pubsub topic '%s' does not exist.
• Unexpected error while serializing PubsubMessage to a byte a
• Failed to create subscription to topic %s on project %s: %s
• Unable to locate declared method.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/992d77e44df73642. Report an issue: GitHub.