{"record":{"id":"08a09c066df4a773","repo":"apache/iceberg","slug":"failed-to-delete-08a09c","errorCode":null,"errorMessage":"Failed to delete {} ({})","messagePattern":"Failed to delete (.+?) \\((.+?)\\)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkCleanupUtil.java","lineNumber":101,"sourceCode":"   * @param files a list of files to delete\n   */\n  public static void deleteFiles(String context, FileIO io, List<? extends ContentFile<?>> files) {\n    List<String> paths = Lists.transform(files, ContentFile::location);\n    if (io instanceof SupportsBulkOperations) {\n      CatalogUtil.deleteFiles(io, paths, \"\");\n    } else {\n      delete(context, io, paths);\n    }\n  }\n\n  private static void delete(String context, FileIO io, List<String> paths) {\n    AtomicInteger deletedFilesCount = new AtomicInteger(0);\n\n    Tasks.foreach(paths)\n        .executeWith(ThreadPools.getWorkerPool())\n        .stopRetryOn(NotFoundException.class)\n        .suppressFailureWhenFinished()\n        .onFailure((path, exc) -> LOG.warn(\"Failed to delete {} ({})\", path, context, exc))\n        .retry(DELETE_NUM_RETRIES)\n        .exponentialBackoff(\n            DELETE_MIN_RETRY_WAIT_MS,\n            DELETE_MAX_RETRY_WAIT_MS,\n            DELETE_TOTAL_RETRY_TIME_MS,\n            2 /* exponential */)\n        .run(\n            path -> {\n              io.deleteFile(path);\n              deletedFilesCount.incrementAndGet();\n            });\n\n    if (deletedFilesCount.get() < paths.size()) {\n      LOG.warn(\"Deleted only {} of {} file(s) ({})\", deletedFilesCount, paths.size(), context);\n    } else {\n      LOG.info(\"Deleted {} file(s) ({})\", paths.size(), context);\n    }\n  }","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkCleanupUtil.java#L83-L119","documentation":"A WARN log from SparkCleanupUtil.delete (via Tasks.foreach onFailure callback) fired when an orphan-file deletion retry loop permanently fails to delete a file. The underlying exception is logged; the failure is suppressed when finished so the commit/abort is not blocked.","triggerScenarios":"SparkCleanupUtil.deleteFiles is called (e.g. on job abort or deleteWhere cleanup) and io.deleteFile throws a retriable exception that still fails after DELETE_NUM_RETRIES, e.g. transient S3 throttling or permission errors.","commonSituations":"Cloud storage rate limiting during large job aborts; credentials lacking delete permission on the table location; files already removed by concurrent compaction.","solutions":["Read the full logged stack trace (exception is passed to the logger) to find the root cause","Verify the table's FileIO credentials allow delete operations","Run remove_orphan_files / rewrite maintenance to clean leftover files","Reduce concurrency or configure cloud storage retries if throttling is the cause"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// verify delete permission before writes\nfileIO.deleteFile(testPath); // or check IAM policy covers s3:DeleteObject on table location","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Grant delete permissions on the entire table location to the write principal","Configure storage-side retries for throttling (S3 retry policy)","Schedule remove_orphan_files to clean files missed by failed cleanups"],"tags":["spark","file-deletion","cleanup","retry"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}