apache/iceberg · warning

Failed to delete file: {}

Error message

Failed to delete file: {}

What it means

DeleteOrphanFilesSparkAction.deleteNonBulk is the fallback for FileIO implementations without bulk support; each file is deleted in parallel via Tasks with noRetry and failure suppression, and per-file failures are logged as 'Failed to delete file: <path>'. The action completes even if some deletes failed, so files may remain.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java:336

  private void deleteBulk(SupportsBulkOperations io, List<String> paths) {
    try {
      io.deleteFiles(paths);
      LOG.info("Deleted {} files using bulk deletes", paths.size());
    } catch (BulkDeletionFailureException e) {
      int deletedFilesCount = paths.size() - e.numberFailedObjects();
      LOG.warn(
          "Deleted only {} of {} files using bulk deletes", deletedFilesCount, paths.size(), e);
    }
  }

  private void deleteNonBulk(List<String> paths) {
    Tasks.Builder<String> deleteTasks =
        Tasks.foreach(paths)
            .noRetry()
            .executeWith(deleteExecutorService)
            .suppressFailureWhenFinished()
            .onFailure((file, exc) -> LOG.warn("Failed to delete file: {}", file, exc));

    if (deleteFunc == null) {
      LOG.info(
          "Table IO {} does not support bulk operations. Using non-bulk deletes.",
          table.io().getClass().getName());
      deleteTasks.run(table.io()::deleteFile);
    } else {
      LOG.info("Custom delete function provided. Using non-bulk deletes");
      deleteTasks.run(deleteFunc::accept);
    }
  }

  @VisibleForTesting
  static Dataset<String> findOrphanFiles(
      Dataset<FileURI> actualFileIdentDS,
      Dataset<FileURI> validFileIdentDS,
      PrefixMismatchMode prefixMismatchMode) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Review the logged exceptions per file and fix the storage-level cause (credentials, permissions)
2. Re-run the RemoveOrphanFiles action after fixing to delete the leftover files
3. Use a FileIO with bulk support (e.g. S3FileIO) for more efficient and resumable deletes

try {
  Actions.forTable(table).deleteOrphanFiles().execute();
} catch (Exception e) {
  // per-file failures were logged; fix and re-run
}

When it happens

Trigger: Any individual table.io().deleteFile(path) (or custom deleteFunc) throwing during a non-bulk orphan-file cleanup — permissions, transient storage errors, or file already removed by another process.

Common situations: Using HadoopFileIO or custom IO without SupportsBulkOperations; concurrent ExpireSnapshots and RemoveOrphanFiles on the same table; read-only mount or missing credentials.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• Failed to delete uncommitted DV {} for table {} task {}
• Deleted only {} of {} files using bulk deletes
• Cannot remove orphan files with an interval less than 24 hou
• Cannot remove orphan files with an interval less than 24 hou
• Cannot remove orphan files with an interval less than 24 hou

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/3ea6054c7573ec74. Report an issue: GitHub.