apache/beam · warning

Failed to delete {} messages due to expired receipt handles.

Error message

Failed to delete {} messages due to expired receipt handles.

What it means

SQS message receipt handles expire (default 60s visibility + lifetime limits). When deleting messages after successful processing, SQS reports which IDs failed due to invalid/expired receipt handles; the reader logs this warning and drops those messages from the pending-delete list — meaning they will reappear after visibility timeout and may be reprocessed.

Source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/sqs/SqsUnboundedReader.java:613

      }

      DeleteMessageBatchResponse result =
          sqsClient.deleteMessageBatch(
              DeleteMessageBatchRequest.builder()
                  .queueUrl(queueUrl())
                  .entries(pendingDeletes.values())
                  .build());

      Map<Boolean, Set<String>> failures =
          result.failed().stream()
              .collect(partitioningBy(this::isHandleInvalid, mapping(e -> e.id(), toSet())));

      // Keep failed IDs only, but discard invalid receipt handles
      pendingDeletes.keySet().retainAll(failures.getOrDefault(FALSE, ImmutableSet.of()));

      int invalidHandles = failures.getOrDefault(TRUE, ImmutableSet.of()).size();
      if (invalidHandles > 0) {
        LOG.warn("Failed to delete {} messages due to expired receipt handles.", invalidHandles);
      }

      retries += 1;
    }
  }

  /** Check {@link BatchResultErrorEntry#code()} for invalid expired receipt handles. */
  private boolean isHandleInvalid(BatchResultErrorEntry error) {
    return RECEIPT_HANDLE_IS_INVALID.equals(error.code());
  }

  /**
   * Messages which have been deleted (via the checkpoint finalize) are no longer in flight. This is
   * only used for flow control and stats.
   */
  private void retire() {
    long nowMsSinceEpoch = now();
    while (true) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Increase the queue's VisibilityTimeout to exceed worst-case per-batch processing time
2. Use SqsIO's configuration to extend visibility/heartbeat if supported by your Beam version
3. Make processing idempotent so redelivered messages are handled safely
4. Reduce delete batch size / issue deletes more frequently to shrink the handle age window

Example fix

// before: visibility too small for processing time
aws sqs set-queue-attributes --queue-url ... --attributes '{"VisibilityTimeout":"30"}'
// after
aws sqs set-queue-attributes --queue-url ... --attributes '{"VisibilityTimeout":"300"}'

// verify receipt handle is still valid before long processing
if (Duration.between(receivedAt, Instant.now()).getSeconds() > visibilityTimeoutSeconds) {
  extendVisibility(client, queueUrl, receiptHandle, visibilityTimeoutSeconds);
}

try {
  client.deleteMessage(b -> b.queueUrl(q).receiptHandle(h));
} catch (ReceiptHandleIsInvalidException e) {
  // handle expired: message was redelivered; rely on idempotent reprocessing
}

When it happens

Trigger: deleteBatch receives a response where some message IDs are reported with an 'expired/invalid receipt handle' failure — happens when processing takes longer than the visibility timeout (or extended via withMaxRetries/visibility settings), so SQS redelivered the message and invalidated the old handle.

Common situations: Long batch processing exceeding VisibilityTimeout; duplicate deliveries; visibility timeout misconfigured much lower than processing time; very large batches of deletes taking too long to issue.

Related errors

• Failed to extend visibility timeout for messages.size() mess
• to(queueUrl) or to(dynamicDestination) is required
• Failed to delete pendingDeletes.size() messages after retrie
• Unable to subscribe to read.queueUrl():
• Ignoring unmatched input fields: {}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/02370c45916b926f. Report an issue: GitHub.