apache/iceberg · warning

Failed to delete file: {}

Error message

Failed to delete file: {}

What it means

When the table's FileIO does not implement SupportsBulkOperations, DeleteOrphanFiles falls back to deleting files one-by-one with Tasks.foreach. Each individual failure is logged with this warning (file path plus exception) and suppressed so the overall cleanup continues.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java:336

  private void deleteBulk(SupportsBulkOperations io, List<String> paths) {
    try {
      io.deleteFiles(paths);
      LOG.info("Deleted {} files using bulk deletes", paths.size());
    } catch (BulkDeletionFailureException e) {
      int deletedFilesCount = paths.size() - e.numberFailedObjects();
      LOG.warn(
          "Deleted only {} of {} files using bulk deletes", deletedFilesCount, paths.size(), e);
    }
  }

  private void deleteNonBulk(List<String> paths) {
    Tasks.Builder<String> deleteTasks =
        Tasks.foreach(paths)
            .noRetry()
            .executeWith(deleteExecutorService)
            .suppressFailureWhenFinished()
            .onFailure((file, exc) -> LOG.warn("Failed to delete file: {}", file, exc));

    if (deleteFunc == null) {
      LOG.info(
          "Table IO {} does not support bulk operations. Using non-bulk deletes.",
          table.io().getClass().getName());
      deleteTasks.run(table.io()::deleteFile);
    } else {
      LOG.info("Custom delete function provided. Using non-bulk deletes");
      deleteTasks.run(deleteFunc::accept);
    }
  }

  @VisibleForTesting
  static Dataset<String> findOrphanFiles(
      Dataset<FileURI> actualFileIdentDS,
      Dataset<FileURI> validFileIdentDS,
      PrefixMismatchMode prefixMismatchMode) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Re-run DeleteOrphanFiles after fixing the underlying cause; already-cleaned files are simply skipped.
2. Grant the job's principal delete permission on the table location (HDFS ACLs or cloud IAM).
3. Switch to a FileIO with bulk support (S3FileIO) for better throughput and consolidated error reporting.
4. If failures cluster on one prefix, check lifecycle rules or concurrent processes competing to delete the same files.

// confirm delete permission before cleanup
io.deleteFile(testPath);

if (!(table.io() instanceof SupportsBulkOperations)) { /* non-bulk path: expect per-file failures */ }

When it happens

Trigger: Running DeleteOrphanFiles with a non-bulk FileIO (e.g. HadoopFileIO) where individual io.deleteFile calls fail — permission denied, file already gone, or transient filesystem errors.

Common situations: HDFS permission errors after permission model changes; files deleted concurrently by another GC run; NFS/S3A transient IOExceptions under load.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Delete failed for {}: {}
• Deleted only {} of {} files using bulk deletes
• Delete failed for {}: {}
• Deleted only {} of {} files using bulk deletes
• Failed to delete file: {}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/41b8e9bfefc292ad. Report an issue: GitHub.