apache/beam · error · IllegalStateException

Received message missing ackId

Error message

Received message missing ackId

What it means

PubsubJsonClient.pull() requires each message in the pull response to carry an ackId used for later acknowledgment. If message.getAckId() is null or empty, IllegalStateException "Received message missing ackId" is thrown. Like the missing publishTime error, this signals a malformed response from the Pub/Sub backend.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubsubJsonClient.java:215

      String dataStr = pubsubMessage.getData();
      byte[] elementBytes = (dataStr == null) ? new byte[0] : pubsubMessage.decodeData();

      // Timestamp.
      long timestampMsSinceEpoch;
      if (timestampAttribute == null || timestampAttribute.isEmpty()) {
        String publishTime = pubsubMessage.getPublishTime();
        if (publishTime == null) {
          throw new IllegalStateException("Received message missing publishTime");
        }
        timestampMsSinceEpoch = parseTimestampAsMsSinceEpoch(publishTime);
      } else {
        timestampMsSinceEpoch = extractTimestampAttribute(timestampAttribute, attributes);
      }

      // Ack id.
      String ackId = message.getAckId();
      if (ackId == null || ackId.isEmpty()) {
        throw new IllegalStateException("Received message missing ackId");
      }

      // Record id, if any.
      @Nullable String recordId = null;
      if (idAttribute != null) {
        recordId = attributes.get(idAttribute);
      }
      if (recordId == null || recordId.isEmpty()) {
        // Fall back to the Pubsub provided message id.
        recordId = checkStateNotNull(pubsubMessage.getMessageId(), "Message ID is missing");
      }

      com.google.pubsub.v1.PubsubMessage.Builder protoMessage =
          com.google.pubsub.v1.PubsubMessage.newBuilder();
      protoMessage.setData(ByteString.copyFrom(elementBytes));
      protoMessage.putAllAttributes(attributes);
      // {@link PubsubMessage} uses `null` or empty string to represent no ordering key.
      // {@link com.google.pubsub.v1.PubsubMessage} does not track string field presence and uses

View on GitHub (pinned to 12126d8942)

Solutions

1. Fix or replace the emulator/proxy so pull responses include ackId for every message.
2. Use the official Google Pub/Sub emulator, which always returns ackId.
3. Verify the Google API client and Beam SDK versions are compatible.
4. Inspect the raw JSON response to confirm which entries are missing ackId.

Example fix

// before (fake emulator response)
{"receivedMessages": [{"message": {"data": "..."}}]}
// after
{"receivedMessages": [{"ackId": "ack-1", "message": {"data": "..."}}]}

for (ReceivedMessage m : response.getReceivedMessagesOrDefault(new ArrayList<>())) {
  if (m.getAckId() == null || m.getAckId().isEmpty()) {
    throw new IllegalStateException("Response entry missing ackId; backend is non-conformant");
  }
}

try {
  incoming = client.pull(request);
} catch (IllegalStateException e) {
  if (e.getMessage() != null && e.getMessage().contains("missing ackId")) {
    // treat backend as broken: reconnect or switch to the official emulator
  }
}

When it happens

Trigger: pull() returning a message entry without an ackId — typically from a non-conformant emulator/proxy response, truncated API responses, or a hand-rolled fake Pub/Sub service used in tests.

Common situations: Custom Pub/Sub emulators in CI, MITM proxies rewriting responses, or newer API client versions interacting badly with the response parser.

Related errors

• Received message missing publishTime
• Pubsub topic '%s' does not exist.
• Unexpected error while serializing PubsubMessage to a byte a
• Failed to create subscription to topic %s on project %s: %s
• Unable to locate declared method.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9707d08ed924539c. Report an issue: GitHub.