{"record":{"id":"199f9635d8749a47","repo":"apache/iceberg","slug":"failed-to-delete-file-199f96","errorCode":null,"errorMessage":"Failed to delete {} file: {}","messagePattern":"Failed to delete (.+?) file: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"core/src/main/java/org/apache/iceberg/CatalogUtil.java","lineNumber":246,"sourceCode":"        LOG.warn(\"Failed to bulk delete {} {} files\", e.numberFailedObjects(), type, e);\n      } catch (RuntimeException e) {\n        LOG.warn(\"Failed to bulk delete {} files\", type, e);\n      }\n    } else {\n      if (concurrent) {\n        concurrentlyDeleteFiles(io, files, type);\n      } else {\n        files.forEach(file -> deleteFile(io, file, type));\n      }\n    }\n  }\n\n  private static void concurrentlyDeleteFiles(FileIO io, Iterable<String> files, String type) {\n    Tasks.foreach(files)\n        .executeWith(ThreadPools.getWorkerPool())\n        .noRetry()\n        .suppressFailureWhenFinished()\n        .onFailure((file, exc) -> LOG.warn(\"Failed to delete {} file: {}\", type, file, exc))\n        .run(io::deleteFile);\n  }\n\n  private static void deleteFile(FileIO io, String file, String type) {\n    try {\n      io.deleteFile(file);\n    } catch (RuntimeException e) {\n      LOG.warn(\"Failed to delete {} file {}\", type, file, e);\n    }\n  }\n\n  /**\n   * Load a custom catalog implementation.\n   *\n   * <p>The catalog must have a no-arg constructor. If the class implements Configurable, a Hadoop\n   * config will be passed using Configurable.setConf. {@link Catalog#initialize(String catalogName,\n   * Map options)} is called to complete the initialization.\n   *","sourceCodeStart":228,"sourceCodeEnd":264,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/CatalogUtil.java#L228-L264","documentation":"A WARN logged per-file by concurrentlyDeleteFiles: when FileIO is not bulk-capable and concurrent=true, each file is deleted in the worker pool via Tasks; a failure on any individual delete is logged (with the type and file path) and suppressed so other deletions proceed. Failed files are left behind as orphans; the overall drop still succeeds.","triggerScenarios":"CatalogUtil.deleteFiles with a plain (non-SupportsBulkOperations) FileIO and concurrent=true; an individual io.deleteFile(file) throws — file already deleted by a concurrent job, permission denied, throttling, or transient storage errors.","commonSituations":"HadoopFileIO/local FS permission issues; two pipelines dropping or expiring the same table concurrently; object store rate limits under the worker pool's parallelism.","solutions":["Check the logged file path and delete it manually or via orphan-file cleanup.","Re-run the cleanup; already-deleted files typically no longer fail (or handle NoSuchKey).","Fix per-prefix IAM/FS permissions for the failing files.","Reduce worker-pool concurrency via ThreadPools configuration if throttling is the cause."],"exampleFix":"// before: drop with implicit concurrent deletes leaking some files\ncatalog.dropTable(identifier);\n// after: sweep any leftovers\ncatalog.dropTable(identifier);\nSparkActions.get().deleteOrphanFiles(spark, tableLocation).olderThan(...).execute();","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { catalog.dropTable(id); } finally { scheduleOrphanSweep(tableLocation); }","preventionTips":["Avoid racing cleanup jobs on the same table","Configure ThreadPools worker-pool size to stay under provider rate limits","Grant delete permissions on the entire table prefix","Treat individual delete warnings as expected under concurrent expiry; sweep afterwards"],"tags":["file-delete","concurrency","orphan-files"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}