{"record":{"id":"d769f55c09d7ca7a","repo":"apache/iceberg","slug":"failed-to-bulk-delete-files","errorCode":null,"errorMessage":"Failed to bulk delete {} {} files","messagePattern":"Failed to bulk delete (.+?) (.+?) files","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"core/src/main/java/org/apache/iceberg/CatalogUtil.java","lineNumber":228,"sourceCode":"  public static void deleteFiles(FileIO io, Iterable<String> files, String type) {\n    deleteFiles(io, files, type, true);\n  }\n\n  /**\n   * Helper to delete files. Bulk deletion is used if possible.\n   *\n   * @param io FileIO for deletes\n   * @param files files to delete\n   * @param type type of files being deleted\n   * @param concurrent controls concurrent deletion. Only applicable for non-bulk FileIO\n   */\n  public static void deleteFiles(\n      FileIO io, Iterable<String> files, String type, boolean concurrent) {\n    if (io instanceof SupportsBulkOperations bulkIO) {\n      try {\n        bulkIO.deleteFiles(files);\n      } catch (BulkDeletionFailureException e) {\n        LOG.warn(\"Failed to bulk delete {} {} files\", e.numberFailedObjects(), type, e);\n      } catch (RuntimeException e) {\n        LOG.warn(\"Failed to bulk delete {} files\", type, e);\n      }\n    } else {\n      if (concurrent) {\n        concurrentlyDeleteFiles(io, files, type);\n      } else {\n        files.forEach(file -> deleteFile(io, file, type));\n      }\n    }\n  }\n\n  private static void concurrentlyDeleteFiles(FileIO io, Iterable<String> files, String type) {\n    Tasks.foreach(files)\n        .executeWith(ThreadPools.getWorkerPool())\n        .noRetry()\n        .suppressFailureWhenFinished()\n        .onFailure((file, exc) -> LOG.warn(\"Failed to delete {} file: {}\", type, file, exc))","sourceCodeStart":210,"sourceCodeEnd":246,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/CatalogUtil.java#L210-L246","documentation":"A WARN log when bulk file deletion via SupportsBulkOperations.deleteFiles throws BulkDeletionFailureException: the object store reported that some (but not necessarily all) of the files could not be deleted; e.numberFailedObjects() says how many. Iceberg logs and continues — deletion is best-effort during table drop, so some files may remain as orphans.","triggerScenarios":"CatalogUtil.deleteFiles is called with a FileIO implementing SupportsBulkOperations (e.g. S3FileIO) and the bulk delete API returns partial failures — objects already gone, permission denied on some keys, throttling (S3 SlowDown), or batch size limits.","commonSituations":"AWS S3 batch delete throttling under high table-drop concurrency; files deleted concurrently by another cleanup job; bucket policies blocking deletes on some prefixes.","solutions":["Retry the drop/cleanup after the transient errors; already-deleted files are skipped by most stores.","Check numberFailedObjects() in the logged exception and delete the remaining paths manually or re-run orphan cleanup.","Review bucket IAM policies and prefix-level deny rules for the failing objects.","Reduce concurrency or add S3 request-rate headroom if throttling (SlowDown) is the cause."],"exampleFix":"// before: assuming drop removes everything\ncatalog.dropTable(identifier);\n// after: verify and sweep\nif (!catalog.tableExists(identifier)) {\n  SparkActions.get().deleteOrphanFiles(spark, tableLocation).execute();\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { catalog.dropTable(id); } catch (RuntimeException e) { /* re-run drop/cleanup; deletions are idempotent */ }","preventionTips":["Re-run cleanup after partial bulk-delete failures — deletes are idempotent","Check exception.numberFailedObjects() to size the leftover set","Ensure bulk delete SDK limits (batch size, rate) are tuned","Use a FileIO whose bulk ops match your object store"],"tags":["bulk-delete","object-store","s3"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}