apache/iceberg · warning

Failed to delete file: {}

Error message

Failed to delete file: {}

What it means

WARN log produced by the Tasks.foreach onFailure callback in deleteNonBulk. When the table's FileIO lacks bulk-delete support, each path is deleted one-by-one via table.io().deleteFile on an executor; any per-file exception (with no retries configured) is logged here instead of failing the job.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java:336

  private void deleteBulk(SupportsBulkOperations io, List<String> paths) {
    try {
      io.deleteFiles(paths);
      LOG.info("Deleted {} files using bulk deletes", paths.size());
    } catch (BulkDeletionFailureException e) {
      int deletedFilesCount = paths.size() - e.numberFailedObjects();
      LOG.warn(
          "Deleted only {} of {} files using bulk deletes", deletedFilesCount, paths.size(), e);
    }
  }

  private void deleteNonBulk(List<String> paths) {
    Tasks.Builder<String> deleteTasks =
        Tasks.foreach(paths)
            .noRetry()
            .executeWith(deleteExecutorService)
            .suppressFailureWhenFinished()
            .onFailure((file, exc) -> LOG.warn("Failed to delete file: {}", file, exc));

    if (deleteFunc == null) {
      LOG.info(
          "Table IO {} does not support bulk operations. Using non-bulk deletes.",
          table.io().getClass().getName());
      deleteTasks.run(table.io()::deleteFile);
    } else {
      LOG.info("Custom delete function provided. Using non-bulk deletes");
      deleteTasks.run(deleteFunc::accept);
    }
  }

  @VisibleForTesting
  static Dataset<String> findOrphanFiles(
      Dataset<FileURI> actualFileIdentDS,
      Dataset<FileURI> validFileIdentDS,
      PrefixMismatchMode prefixMismatchMode) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Read the logged exception to identify the failing file and root cause (ACLs, absence, throttling).
2. Re-run ExpireSnapshots/RemoveOrphanFiles — deleting an already-missing file is expected to fail harmlessly and remaining files are retried.
3. Fix filesystem permissions or object-lock/lifecycle settings for the affected paths.
4. Switch to a FileIO supporting bulk deletes (e.g. S3FileIO) so deletes go through the bulk path with better parallelism.

Example fix

// before: silent assumption deletes succeed
tasks.run(table.io()::deleteFile);
// after: optionally supply a tolerant delete function
action.deleteWith(path -> {
  try { io.deleteFile(path); } catch (Exception e) { LOG.warn("skip {}: {}", path, e.toString()); }
});

// ensure delete permissions before the action
table.io().deleteFile(testPath);

if (!(table.io() instanceof SupportsBulkOperations)) { /* non-bulk path: expect per-file warnings */ }

try { io.deleteFile(path); } catch (Exception e) { LOG.warn("Failed to delete file: {}", path, e); failedPaths.add(path); }

When it happens

Trigger: DeleteOrphanFilesSparkAction.deleteFiles with a FileIO that does not implement SupportsBulkOperations (or an explicit deleteFunc), and io.deleteFile(path) throws for a specific file — permissions, NoSuchKey races, network errors.

Common situations: Local/Hadoop file systems or custom FileIOs without bulk support; files already removed by a concurrent job; permission or lifecycle-policy blocks on individual objects.

Related errors

• Deleted only {} of {} files using bulk deletes
• Failed to scan delegate parser in {}:
• Delete failed for {}: {}
• Failed to delete file: {}
• Delete failed for {}: {}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/9743576840803453. Report an issue: GitHub.