{"record":{"id":"6f67309a913bec1c","repo":"apache/iceberg","slug":"encountered-failure-when-deleting-batch","errorCode":null,"errorMessage":"Encountered failure when deleting batch","messagePattern":"Encountered failure when deleting batch","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java","lineNumber":320,"sourceCode":"        keysToDelete.stream()\n            .map(key -> ObjectIdentifier.builder().key(key).build())\n            .collect(Collectors.toList());\n    DeleteObjectsRequest request =\n        DeleteObjectsRequest.builder()\n            .bucket(bucket)\n            .delete(Delete.builder().objects(objectIds).build())\n            .build();\n    List<String> failures = Lists.newArrayList();\n    try {\n      DeleteObjectsResponse response = client.s3().deleteObjects(request);\n      if (response.hasErrors()) {\n        failures.addAll(\n            response.errors().stream()\n                .map(error -> String.format(\"s3://%s/%s\", request.bucket(), error.key()))\n                .collect(Collectors.toList()));\n      }\n    } catch (Exception e) {\n      LOG.warn(\"Encountered failure when deleting batch\", e);\n      failures.addAll(\n          request.delete().objects().stream()\n              .map(obj -> String.format(\"s3://%s/%s\", request.bucket(), obj.key()))\n              .collect(Collectors.toList()));\n    }\n    return failures;\n  }\n\n  @Override\n  public Iterable<FileInfo> listPrefix(String prefix) {\n    PrefixedS3Client client = clientForStoragePath(prefix);\n\n    S3URI uri = new S3URI(prefix, client.s3FileIOProperties().bucketToAccessPointMapping());\n    if (uri.useS3DirectoryBucket()\n        && client.s3FileIOProperties().isS3DirectoryBucketListPrefixAsDirectory()) {\n      uri = uri.toDirectoryPath();\n    }\n","sourceCodeStart":302,"sourceCodeEnd":338,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java#L302-L338","documentation":"S3FileIO.deleteBatch() wraps the S3 client's batch deleteObjects call. Any exception from the AWS SDK (network failure, throttling, auth error) is caught, logged with this message, and all keys in the batch are added to the failures list so callers can retry them. The original exception is attached to the log entry.","triggerScenarios":"deleteFiles() with batches large enough to trigger S3 throttling; expired credentials mid-run; network interruption during DeleteObjects; SDK client misconfiguration.","commonSituations":"Very high delete throughput causing 503 slow-down responses; STS credentials expiring during long compaction jobs; connectivity issues in on-prem/MinIO S3-compatible storage.","solutions":["Retry the failed paths — they are returned/logged for exactly this purpose","Check the attached exception: 503/SlowDown means reduce batch size or add backoff","Refresh or widen credential lifetime (session token expiry)","Verify S3 endpoint/region configuration, especially with custom endpoints"],"exampleFix":"// before: huge batches with no tuning\nprops.put(\"s3.delete.batch-size\", \"5000\");\n// after: smaller batch reduces throttling\nprops.put(\"s3.delete.batch-size\", \"256\");","handlingStrategy":"retry","validationCode":"// pre-check credentials and endpoint reachability\ns3Client.headBucket(b -> b.bucket(bucket));","typeGuard":null,"tryCatchPattern":"try { io.deleteFiles(paths); } catch (RuntimeException e) { /* batch-level SDK errors are logged; failed keys are recoverable */ retryLater(paths); }","preventionTips":["Retry the failed keys — deleteBatch already collects them into a failures list","Reduce delete batch size under throttling (503 SlowDown)","Use long-lived or auto-refreshing credentials for long compaction jobs"],"tags":["aws","s3","network","batch-delete"],"backgroundTag":"network-request-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}