{"record":{"id":"4af20b559f21450a","repo":"apache/beam","slug":"failed-to-delete-pendingdeletes-size-messages-after-retries","errorCode":null,"errorMessage":"Failed to delete pendingDeletes.size() messages after retries retries","messagePattern":"Failed to delete pendingDeletes\\.size\\(\\) messages after retries retries","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/sqs/SqsUnboundedReader.java","lineNumber":589,"sourceCode":"   * <p>CAUTION: May be invoked from a separate thread.\n   */\n  private void deleteBatch(List<String> receiptHandles) throws IOException {\n    int retries = 0;\n\n    FunctionWithIndex<String, DeleteMessageBatchRequestEntry> buildEntry =\n        (handle, id) ->\n            DeleteMessageBatchRequestEntry.builder()\n                .id(Long.toString(id))\n                .receiptHandle(handle)\n                .build();\n\n    Map<String, DeleteMessageBatchRequestEntry> pendingDeletes =\n        mapWithIndex(receiptHandles.stream(), buildEntry).collect(toMap(e -> e.id(), identity()));\n\n    while (!pendingDeletes.isEmpty()) {\n\n      if (retries >= BATCH_OPERATION_MAX_RETIRES) {\n        throw new IOException(\n            \"Failed to delete \"\n                + pendingDeletes.size()\n                + \" messages after \"\n                + retries\n                + \" retries\");\n      }\n\n      DeleteMessageBatchResponse result =\n          sqsClient.deleteMessageBatch(\n              DeleteMessageBatchRequest.builder()\n                  .queueUrl(queueUrl())\n                  .entries(pendingDeletes.values())\n                  .build());\n\n      Map<Boolean, Set<String>> failures =\n          result.failed().stream()\n              .collect(partitioningBy(this::isHandleInvalid, mapping(e -> e.id(), toSet())));\n","sourceCodeStart":571,"sourceCodeEnd":607,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/sqs/SqsUnboundedReader.java#L571-L607","documentation":"SqsUnboundedReader.deleteBatch retries deleting message batches up to BATCH_OPERATION_MAX_RETIRES; if pendingDeletes is still non-empty after exhausting retries it throws IOException 'Failed to delete N messages after R retries'. Messages were read but their receipt handles could not be deleted from the queue in time.","triggerScenarios":"SQS DeleteMessageBatch repeatedly failing/partially failing (transient AWS errors, invalid/expired receipt handles, throttling) across all retry attempts while the reader keeps retrying.","commonSituations":"Long-running reads where visibility timeout expired and receipt handles became stale; SQS throttling under high throughput; temporary AWS outage/credential issues during deletes.","solutions":["Check the SQS queue permissions and credentials for DeleteMessageBatch","Reduce read throughput / increase SQS queue capacity to avoid throttling","Ensure messages are deleted within the visibility timeout window","Inspect AWS-side failures (CloudWatch metrics) and transient errors; messages will reappear after visibility timeout so downstream processing must be idempotent"],"exampleFix":"// before\n// visibility timeout of 30s with batch processing taking minutes\n// after\n// changeVisibilityTimeout(queueUrl, Duration.ofMinutes(10));\n// or process/delete batches well within the visibility window","handlingStrategy":"retry","validationCode":"// pre-check queue accessibility before reading\nsqs.getQueueUrl(GetQueueUrlRequest.builder().queueName(name).build());","typeGuard":null,"tryCatchPattern":"try { reader.deleteBatch(handles); } catch (IOException e) { /* messages will reappear after visibility timeout; ensure idempotent processing */ }","preventionTips":["Make message processing idempotent since failed deletes cause redelivery","Keep visibility timeout comfortably larger than batch processing time","Monitor SQS NumberOfMessagesDeleted and throttling metrics","Verify IAM permissions for sqs:DeleteMessageBatch"],"tags":["java","sqs","aws","retry"],"backgroundTag":"http-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}