apache/beam · error · IOException

Failed to extend visibility timeout for messages.size() mess

Error message

Failed to extend visibility timeout for messages.size() messages after retries retries

What it means

SqsUnboundedReader throws this IOException when a batch ChangeMessageVisibility request to SQS fails to extend visibility timeouts for all messages even after BATCH_OPERATION_MAX_RETIRES retries. Messages whose visibility expires will be redelivered, so the reader aborts rather than silently lose delivery guarantees.

Source

Thrown at sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/sqs/SqsUnboundedReader.java:827

  void extendBatch(long nowMsSinceEpoch, List<KV<String, String>> messages, int extensionSec)
      throws IOException {
    int retries = 0;

    Function<KV<String, String>, ChangeMessageVisibilityBatchRequestEntry> buildEntry =
        kv ->
            ChangeMessageVisibilityBatchRequestEntry.builder()
                .visibilityTimeout(extensionSec)
                .id(kv.getKey())
                .receiptHandle(kv.getValue())
                .build();

    Map<String, ChangeMessageVisibilityBatchRequestEntry> pendingExtends =
        messages.stream().collect(toMap(KV::getKey, buildEntry));

    while (!pendingExtends.isEmpty()) {

      if (retries >= BATCH_OPERATION_MAX_RETIRES) {
        throw new IOException(
            "Failed to extend visibility timeout for "
                + messages.size()
                + " messages after "
                + retries
                + " retries");
      }

      ChangeMessageVisibilityBatchResponse response =
          sqsClient.changeMessageVisibilityBatch(
              ChangeMessageVisibilityBatchRequest.builder()
                  .queueUrl(queueUrl())
                  .entries(pendingExtends.values())
                  .build());

      Map<Boolean, Set<String>> failures =
          response.failed().stream()
              .collect(partitioningBy(this::isHandleInvalid, mapping(e -> e.id(), toSet())));

View on GitHub (pinned to 12126d8942)

Solutions

Verify the AWS credentials/IAM policy grant sqs:ChangeMessageVisibility on the queue.
Check for SQS throttling (ThrottledException) and reduce pipeline parallelism or add backoff.
Ensure no other consumer is deleting the same messages concurrently (duplicate workers sharing a queue).
Increase retries headroom or batch sizes, and check queue visibility timeout settings relative to batch processing time.

Example fix

// before
throw new IOException("Failed to extend visibility timeout for " + messages.size() + " messages after " + retries + " retries");
// after
// inspect per-entry failures before giving up, and log/omit only truly failed message IDs
for (ChangeMessageVisibilityBatchResult r : results) {
  failedIds.addAll(r.getFailed()); // surface which messages failed and why
}

Defensive patterns

Strategy: retry

Validate before calling

// Pre-flight: ensure IAM policy allows sqs:ChangeMessageVisibility on the queue
aws sqs get-queue-attributes --queue-url $QUEUE_URL --attribute-names Policy

Try / catch

try { reader.advance(); } catch (IOException e) {
  if (e.getMessage().contains("Failed to extend visibility timeout")) {
    // backoff and restart the source; messages will be redelivered
    Thread.sleep(TimeUnit.MINUTES.toMillis(1));
  }
}

Prevention

Grant sqs:ChangeMessageVisibility to runner worker roles
Set queue visibility timeout comfortably above batch processing time
Monitor SQS ThrottledException metrics and add client-side rate limiting
Avoid multiple consumers sharing the same queue

When it happens

Trigger: SQS changeMessageVisibilityBatch repeatedly returns per-message failures (or throttles/errors) for the remaining pendingExtends entries across all retry attempts in SqsUnboundedReader.

Common situations: SQS throttling under high throughput, messages already deleted or processed by another consumer, IAM policy lacking changeMessageVisibility permission, messages past the 12-hour total visibility extension limit.

Understand the failure class

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8312dafea5a06746. Report an issue: GitHub.