apache/iceberg · warning

Failed to delete file

Error message

Failed to delete {} file: {}

What it means

A WARN logged per-file by concurrentlyDeleteFiles: when FileIO is not bulk-capable and concurrent=true, each file is deleted in the worker pool via Tasks; a failure on any individual delete is logged (with the type and file path) and suppressed so other deletions proceed. Failed files are left behind as orphans; the overall drop still succeeds.

Solutions

1. Check the logged file path and delete it manually or via orphan-file cleanup.
2. Re-run the cleanup; already-deleted files typically no longer fail (or handle NoSuchKey).
3. Fix per-prefix IAM/FS permissions for the failing files.
4. Reduce worker-pool concurrency via ThreadPools configuration if throttling is the cause.

Example fix

// before: drop with implicit concurrent deletes leaking some files
catalog.dropTable(identifier);
// after: sweep any leftovers
catalog.dropTable(identifier);
SparkActions.get().deleteOrphanFiles(spark, tableLocation).olderThan(...).execute();

try { catalog.dropTable(id); } finally { scheduleOrphanSweep(tableLocation); }

When it happens

Trigger: CatalogUtil.deleteFiles with a plain (non-SupportsBulkOperations) FileIO and concurrent=true; an individual io.deleteFile(file) throws — file already deleted by a concurrent job, permission denied, throttling, or transient storage errors.

Common situations: HadoopFileIO/local FS permission issues; two pipelines dropping or expiring the same table concurrently; object store rate limits under the worker pool's parallelism.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to delete file
• Cannot call acquireLock twice for
• Cannot commit: Base metadata location
• Cannot commit changes based on stale table metadata
• Cannot commit file that conflicts with existing partition

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/199f9635d8749a47. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/CatalogUtil.java:246

        LOG.warn("Failed to bulk delete {} {} files", e.numberFailedObjects(), type, e);
      } catch (RuntimeException e) {
        LOG.warn("Failed to bulk delete {} files", type, e);
      }
    } else {
      if (concurrent) {
        concurrentlyDeleteFiles(io, files, type);
      } else {
        files.forEach(file -> deleteFile(io, file, type));
      }
    }
  }

  private static void concurrentlyDeleteFiles(FileIO io, Iterable<String> files, String type) {
    Tasks.foreach(files)
        .executeWith(ThreadPools.getWorkerPool())
        .noRetry()
        .suppressFailureWhenFinished()
        .onFailure((file, exc) -> LOG.warn("Failed to delete {} file: {}", type, file, exc))
        .run(io::deleteFile);
  }

  private static void deleteFile(FileIO io, String file, String type) {
    try {
      io.deleteFile(file);
    } catch (RuntimeException e) {
      LOG.warn("Failed to delete {} file {}", type, file, e);
    }
  }

  /**
   * Load a custom catalog implementation.
   *
   * <p>The catalog must have a no-arg constructor. If the class implements Configurable, a Hadoop
   * config will be passed using Configurable.setConf. {@link Catalog#initialize(String catalogName,
   * Map options)} is called to complete the initialization.
   *

View on GitHub (pinned to 86d9c8fc54)