apache/iceberg · warning

Failed to delete {} ({})

Error message

Failed to delete {} ({})

What it means

A WARN log from SparkCleanupUtil.delete (via Tasks.foreach onFailure callback) fired when an orphan-file deletion retry loop permanently fails to delete a file. The underlying exception is logged; the failure is suppressed when finished so the commit/abort is not blocked.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkCleanupUtil.java:101

   * @param files a list of files to delete
   */
  public static void deleteFiles(String context, FileIO io, List<? extends ContentFile<?>> files) {
    List<String> paths = Lists.transform(files, ContentFile::location);
    if (io instanceof SupportsBulkOperations) {
      CatalogUtil.deleteFiles(io, paths, "");
    } else {
      delete(context, io, paths);
    }
  }

  private static void delete(String context, FileIO io, List<String> paths) {
    AtomicInteger deletedFilesCount = new AtomicInteger(0);

    Tasks.foreach(paths)
        .executeWith(ThreadPools.getWorkerPool())
        .stopRetryOn(NotFoundException.class)
        .suppressFailureWhenFinished()
        .onFailure((path, exc) -> LOG.warn("Failed to delete {} ({})", path, context, exc))
        .retry(DELETE_NUM_RETRIES)
        .exponentialBackoff(
            DELETE_MIN_RETRY_WAIT_MS,
            DELETE_MAX_RETRY_WAIT_MS,
            DELETE_TOTAL_RETRY_TIME_MS,
            2 /* exponential */)
        .run(
            path -> {
              io.deleteFile(path);
              deletedFilesCount.incrementAndGet();
            });

    if (deletedFilesCount.get() < paths.size()) {
      LOG.warn("Deleted only {} of {} file(s) ({})", deletedFilesCount, paths.size(), context);
    } else {
      LOG.info("Deleted {} file(s) ({})", paths.size(), context);
    }
  }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Read the full logged stack trace (exception is passed to the logger) to find the root cause
2. Verify the table's FileIO credentials allow delete operations
3. Run remove_orphan_files / rewrite maintenance to clean leftover files
4. Reduce concurrency or configure cloud storage retries if throttling is the cause

// verify delete permission before writes
fileIO.deleteFile(testPath); // or check IAM policy covers s3:DeleteObject on table location

When it happens

Trigger: SparkCleanupUtil.deleteFiles is called (e.g. on job abort or deleteWhere cleanup) and io.deleteFile throws a retriable exception that still fails after DELETE_NUM_RETRIES, e.g. transient S3 throttling or permission errors.

Common situations: Cloud storage rate limiting during large job aborts; credentials lacking delete permission on the table location; files already removed by concurrent compaction.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Failed to delete {} ({})
• Deleted only {} of {} file(s) ({})
• Failed to delete {} ({})
• Deleted only {} of {} file(s) ({})
• Failed to delete: {}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/08a09c066df4a773. Report an issue: GitHub.