What causes "Failed to delete {} ({})" in apache/iceberg?

Check the embedded exception for the root cause (permissions, throttling, network).

Failed to delete ( )

Error message

Failed to delete {} ({})

What it means

SparkCleanupUtil.deleteFiles deletes files in parallel via Tasks.foreach with retries and exponential backoff. When a file still fails to delete after all retries (excluding NotFoundException, which is treated as success), it logs which path failed and why. This typically happens during abort cleanup of a Spark write job; leftover files usually have no impact on table correctness.

Solutions

Check the embedded exception for the root cause (permissions, throttling, network).
Manually delete orphaned files afterward with remove_orphan_files.
Retry the aborted job; deletion is retried internally with backoff, but persistent issues need the storage problem fixed.

Defensive patterns

Strategy: retry

Validate before calling

boolean canDelete = io.deleteFile != null; // ensure credentials/permissions valid before job start

Try / catch

try { io.deleteFile(path); } catch (UncheckedIOException e) { log.warn("Leftover file {} to clean later", path, e); }

Prevention

Verify write/delete permissions for the job identity before large writes.
Avoid delete storms that trigger object-store throttling; pace concurrency.
Run remove_orphan_files on a schedule to sweep leftovers.

When it happens

Trigger: A transient or persistent storage error (permissions, throttling, network) during abort-time deletion of uncommitted data files; retried DELETE_NUM_RETRIES times with backoff before this warning.

Common situations: S3/GCS rate limiting (429/503) under heavy parallel deletes; stale credentials or revoked permissions on the job's identity; concurrent cleanup by another process already removed files with a non-NotFoundException error.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/6b0f1e646982b65e. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkCleanupUtil.java:101

   * @param files a list of files to delete
   */
  public static void deleteFiles(String context, FileIO io, List<? extends ContentFile<?>> files) {
    List<String> paths = Lists.transform(files, ContentFile::location);
    if (io instanceof SupportsBulkOperations) {
      CatalogUtil.deleteFiles(io, paths, "");
    } else {
      delete(context, io, paths);
    }
  }

  private static void delete(String context, FileIO io, List<String> paths) {
    AtomicInteger deletedFilesCount = new AtomicInteger(0);

    Tasks.foreach(paths)
        .executeWith(ThreadPools.getWorkerPool())
        .stopRetryOn(NotFoundException.class)
        .suppressFailureWhenFinished()
        .onFailure((path, exc) -> LOG.warn("Failed to delete {} ({})", path, context, exc))
        .retry(DELETE_NUM_RETRIES)
        .exponentialBackoff(
            DELETE_MIN_RETRY_WAIT_MS,
            DELETE_MAX_RETRY_WAIT_MS,
            DELETE_TOTAL_RETRY_TIME_MS,
            2 /* exponential */)
        .run(
            path -> {
              io.deleteFile(path);
              deletedFilesCount.incrementAndGet();
            });

    if (deletedFilesCount.get() < paths.size()) {
      LOG.warn("Deleted only {} of {} file(s) ({})", deletedFilesCount, paths.size(), context);
    } else {
      LOG.info("Deleted {} file(s) ({})", paths.size(), context);
    }
  }

View on GitHub (pinned to 86d9c8fc54)