What causes "Failed to bulk delete {} {} files" in apache/iceberg?

Retry the drop/cleanup after the transient errors; already-deleted files are skipped by most stores.

Failed to bulk delete {} {} files

Error message

Failed to bulk delete {} {} files

What it means

A WARN log when bulk file deletion via SupportsBulkOperations.deleteFiles throws BulkDeletionFailureException: the object store reported that some (but not necessarily all) of the files could not be deleted; e.numberFailedObjects() says how many. Iceberg logs and continues — deletion is best-effort during table drop, so some files may remain as orphans.

Source

Thrown at core/src/main/java/org/apache/iceberg/CatalogUtil.java:228

  public static void deleteFiles(FileIO io, Iterable<String> files, String type) {
    deleteFiles(io, files, type, true);
  }

  /**
   * Helper to delete files. Bulk deletion is used if possible.
   *
   * @param io FileIO for deletes
   * @param files files to delete
   * @param type type of files being deleted
   * @param concurrent controls concurrent deletion. Only applicable for non-bulk FileIO
   */
  public static void deleteFiles(
      FileIO io, Iterable<String> files, String type, boolean concurrent) {
    if (io instanceof SupportsBulkOperations bulkIO) {
      try {
        bulkIO.deleteFiles(files);
      } catch (BulkDeletionFailureException e) {
        LOG.warn("Failed to bulk delete {} {} files", e.numberFailedObjects(), type, e);
      } catch (RuntimeException e) {
        LOG.warn("Failed to bulk delete {} files", type, e);
      }
    } else {
      if (concurrent) {
        concurrentlyDeleteFiles(io, files, type);
      } else {
        files.forEach(file -> deleteFile(io, file, type));
      }
    }
  }

  private static void concurrentlyDeleteFiles(FileIO io, Iterable<String> files, String type) {
    Tasks.foreach(files)
        .executeWith(ThreadPools.getWorkerPool())
        .noRetry()
        .suppressFailureWhenFinished()
        .onFailure((file, exc) -> LOG.warn("Failed to delete {} file: {}", type, file, exc))

View on GitHub (pinned to 86d9c8fc54)

Solutions

Retry the drop/cleanup after the transient errors; already-deleted files are skipped by most stores.
Check numberFailedObjects() in the logged exception and delete the remaining paths manually or re-run orphan cleanup.
Review bucket IAM policies and prefix-level deny rules for the failing objects.
Reduce concurrency or add S3 request-rate headroom if throttling (SlowDown) is the cause.

Example fix

// before: assuming drop removes everything
catalog.dropTable(identifier);
// after: verify and sweep
if (!catalog.tableExists(identifier)) {
  SparkActions.get().deleteOrphanFiles(spark, tableLocation).execute();
}

Defensive patterns

Strategy: retry

Try / catch

try { catalog.dropTable(id); } catch (RuntimeException e) { /* re-run drop/cleanup; deletions are idempotent */ }

Prevention

Re-run cleanup after partial bulk-delete failures — deletes are idempotent
Check exception.numberFailedObjects() to size the leftover set
Ensure bulk delete SDK limits (batch size, rate) are tuned
Use a FileIO whose bulk ops match your object store

When it happens

Trigger: CatalogUtil.deleteFiles is called with a FileIO implementing SupportsBulkOperations (e.g. S3FileIO) and the bulk delete API returns partial failures — objects already gone, permission denied on some keys, throttling (S3 SlowDown), or batch size limits.

Common situations: AWS S3 batch delete throttling under high table-drop concurrency; files deleted concurrently by another cleanup job; bucket policies blocking deletes on some prefixes.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/d769f55c09d7ca7a. Report an issue: GitHub.