Caught unexpected exception during batch deletion:

Error message

Caught unexpected exception during batch deletion:

What it means

S3FileIO.deleteFiles submits batch deletion tasks and, when collecting results, a task failing with an ExecutionException is not rethrown — the cause is logged at WARN with 'Caught unexpected exception during batch deletion: ' and counted implicitly via failed deletions. This means objects may remain in S3 after deleteFiles returns; the error is surfaced only in logs, so silent data retention is possible.

Source

Thrown at aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java:260

      for (Map.Entry<String, Collection<String>> bucketToObjectsEntry :
          bucketToObjects.asMap().entrySet()) {
        String bucket = bucketToObjectsEntry.getKey();
        Collection<String> keys = bucketToObjectsEntry.getValue();
        Future<List<String>> deletionTask =
            executorService()
                .submit(() -> deleteBatch(clientForStoragePath("s3://" + bucket), bucket, keys));
        deletionTasks.add(deletionTask);
      }

      int totalFailedDeletions = 0;

      for (Future<List<String>> deletionTask : deletionTasks) {
        try {
          List<String> failedDeletions = deletionTask.get();
          failedDeletions.forEach(path -> LOG.warn("Failed to delete object at path {}", path));
          totalFailedDeletions += failedDeletions.size();
        } catch (ExecutionException e) {
          LOG.warn("Caught unexpected exception during batch deletion: ", e.getCause());
        } catch (InterruptedException e) {
          Thread.currentThread().interrupt();
          deletionTasks.stream().filter(task -> !task.isDone()).forEach(task -> task.cancel(true));
          throw new RuntimeException("Interrupted when waiting for deletions to complete", e);
        }
      }

      if (totalFailedDeletions > 0) {
        throw new BulkDeletionFailureException(totalFailedDeletions);
      }
    }
  }

  private void tagFileToDelete(PrefixedS3Client client, String path, Set<Tag> deleteTags)
      throws S3Exception {
    S3URI location = new S3URI(path, client.s3FileIOProperties().bucketToAccessPointMapping());
    String bucket = location.bucket();
    String objectKey = location.key();

View on GitHub (pinned to 86d9c8fc54)

Solutions

Inspect application logs for the WARN entry and its cause to identify failed object keys
Re-run deleteFiles after fixing the underlying cause (permissions, throttling) — deletion is idempotent for already-deleted objects
Increase batch deletion parallelism/limits or add retry/backoff for SlowDown errors
Grant s3:DeleteObject on the full prefix in the bucket policy / IAM role

Example fix

// before: assuming all deletes succeeded
fileIO.deleteFiles(tasks);
// after: verify via logs or re-check
fileIO.deleteFiles(tasks);
LOG.info("Batch deletion finished; check logs for 'Caught unexpected exception during batch deletion' and re-run for leftovers");

Defensive patterns

Strategy: retry

Validate before calling

// pre-check permissions/scope before mass deletion
deleteCandidates.forEach(key ->
  require(key.startsWith("s3://" + bucket + "/" + allowedPrefix), "key outside allowed prefix: " + key));

Try / catch

List<String> leftovers = fileIO.deleteFiles(tasks);
if (!leftovers.isEmpty()) {
  // re-attempt idempotent deletion after fixing cause (IAM, throttling)
  leftovers.forEach(path -> LOG.error("object not deleted: {}", path));
}
// and monitor logs for "Caught unexpected exception during batch deletion"

Prevention

Grant s3:DeleteObject on the entire prefix used by the table
Tune batch size/parallelism to avoid S3 SlowDown throttling
Re-run deleteFiles after failures — object deletion is idempotent
Alert on the WARN log message during bulk cleanup jobs

When it happens

Trigger: Concurrent S3 batch deletion (DeleteObjects calls via Tasks.groupby) where a worker task throws — e.g. S3 access denied, throttling (SlowDown), missing object permissions, or an SDK client failure — inside one of the deletionTasks futures.

Common situations: Deleting large manifest batches with credentials lacking s3:DeleteObject on some keys; S3 rate limiting during mass deletes; transient network/SDK errors mid-batch; expired session credentials partway through a long cleanup.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/1c72e4403cc9213a. Report an issue: GitHub.