apache/beam · error · IllegalArgumentException

Pubsub message must contain a non-empty payload or at least

Error message

Pubsub message must contain a non-empty payload or at least one attribute.

What it means

Pub/Sub requires every message to carry either non-empty data or at least one attribute; validatePubsubMessage throws IllegalArgumentException when both payload size is 0 and the attribute map is null/empty. This is an early client-side validation of a Pub/Sub API constraint.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PreparePubsubWriteDoFn.java:92

    int totalSize = payloadSize;

    @Nullable String orderingKey = message.getOrderingKey();
    if (orderingKey != null) {
      int orderingKeySize = orderingKey.getBytes(StandardCharsets.UTF_8).length;
      if (orderingKeySize > ORDERING_KEY_MAX_BYTE_SIZE) {
        throw new SizeLimitExceededException(
            "Pubsub message ordering key of length "
                + orderingKeySize
                + " exceeds maximum of "
                + ORDERING_KEY_MAX_BYTE_SIZE
                + " bytes. See https://cloud.google.com/pubsub/quotas#resource_limits");
      }
      totalSize += orderingKeySize;
    }

    final @Nullable Map<String, String> attributes = message.getAttributeMap();
    if (payloadSize == 0 && (attributes == null || attributes.isEmpty())) {
      throw new IllegalArgumentException(
          "Pubsub message must contain a non-empty payload or at least one attribute.");
    }

    if (attributes != null) {
      if (attributes.size() > PUBSUB_MESSAGE_MAX_ATTRIBUTES) {
        throw new SizeLimitExceededException(
            "Pubsub message contains "
                + attributes.size()
                + " attributes which exceeds the maximum of "
                + PUBSUB_MESSAGE_MAX_ATTRIBUTES
                + ". See https://cloud.google.com/pubsub/quotas#resource_limits");
      }

      // Consider attribute encoding overhead, so it doesn't go over the request limits
      totalSize += attributes.size() * PUBSUB_MESSAGE_ATTRIBUTE_ENCODE_ADDITIONAL_BYTES;

      for (Map.Entry<String, String> attribute : attributes.entrySet()) {
        String key = attribute.getKey();

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure every message has a non-empty payload or set at least one attribute (e.g. event type)
2. Add a guard at message-construction time to reject empty payload+attribute combos
3. If tombstones are intended, put a marker attribute like {"deleted":"true"}
4. Fix upstream serialization so null/empty values don't produce zero-length payloads

Example fix

// before
new PubsubMessage(new byte[0], null);
// after
Map<String, String> attrs = attrs != null ? attrs : new HashMap<>();
attrs.putIfAbsent("event-type", "tombstone");
new PubsubMessage(payload == null ? new byte[0] : payload, attrs);

java
if ((payload == null || payload.length == 0)
    && (attrs == null || attrs.isEmpty())) {
  throw new IllegalArgumentException("PubsubMessage needs non-empty payload or at least one attribute");
}

java
try {
  validatePubsubMessage(msg, maxBatchSize);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("non-empty payload or at least one attribute")) {
    msg = new PubsubMessage(msg.getPayload(), addMarkerAttribute(msg.getAttributeMap()));
  } else throw e;
}

When it happens

Trigger: Calling validatePubsubMessage (via PreparePubsubWriteDoFn.process) with a PubsubMessage constructed from an empty byte[] payload and no attributes — e.g. an empty record or a failed serialization producing zero bytes.

Common situations: Downstream DoFn emitting new PubsubMessage(new byte[0], null); a transformer dropping all fields; encoding logic returning empty arrays for null input; deliberately sending 'tombstone' messages without payload and forgetting attributes.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Illegal project name specified in Pubsub subscription: {proj
• Pubsub object name is shorter than 3 characters: {name}
• Pubsub object name is longer than 255 characters: {name}
• Pubsub topic '%s' does not exist.
• Pipeline update will not be possible because the following t

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/74d5e73ff8411fa7. Report an issue: GitHub.