{"record":{"id":"489cc1140edcc353","repo":"apache/iceberg","slug":"bulk-deletion-failed","errorCode":null,"errorMessage":"Bulk deletion failed","messagePattern":"Bulk deletion failed","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"core/src/main/java/org/apache/iceberg/FileCleanupStrategy.java","lineNumber":119,"sourceCode":"          .build();\n    } else {\n      return CloseableIterable.withNoopClose(snapshot.allManifests(fileIO));\n    }\n  }\n\n  protected void deleteFiles(Set<String> pathsToDelete, String fileType) {\n    if (deleteFunc == null && fileIO instanceof SupportsBulkOperations) {\n      try {\n        ((SupportsBulkOperations) fileIO).deleteFiles(pathsToDelete);\n      } catch (BulkDeletionFailureException e) {\n        LOG.warn(\n            \"Bulk deletion failed for {} of {} {} file(s)\",\n            e.numberFailedObjects(),\n            pathsToDelete.size(),\n            fileType,\n            e);\n      } catch (RuntimeException e) {\n        LOG.warn(\"Bulk deletion failed\", e);\n      }\n    } else {\n      Consumer<String> deleteFuncToUse = deleteFunc == null ? defaultDeleteFunc : deleteFunc;\n\n      Tasks.foreach(pathsToDelete)\n          .executeWith(deleteExecutorService)\n          .retry(3)\n          .stopRetryOn(NotFoundException.class)\n          .stopOnFailure()\n          .suppressFailureWhenFinished()\n          .onFailure(\n              (file, thrown) -> LOG.warn(\"Delete failed for {} file: {}\", fileType, file, thrown))\n          .run(deleteFuncToUse::accept);\n    }\n  }\n\n  protected boolean hasAnyStatisticsFiles(TableMetadata tableMetadata) {\n    return !tableMetadata.statisticsFiles().isEmpty()","sourceCodeStart":101,"sourceCodeEnd":137,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/FileCleanupStrategy.java#L101-L137","documentation":"FileCleanupStrategy.deleteFiles first attempts a bulk delete API (e.g. S3 deleteObjects) via Tasks; if that bulk call fails outright it logs 'Bulk deletion failed' at WARN. The failure is logged and swallowed so the cleanup/expire operation itself does not abort; un-deleted files may be left behind as orphans.","triggerScenarios":"Table maintenance (expireSnapshots, deleteFiles/orphan cleanup) calls deleteFiles with enough paths to take the bulk-deletion path, and the bulk delete call throws (IOException, rate limiting, ObjectStore permission error, bulk-delete not supported by the FileIO). Also hit when a custom FileIO's bulk delete implementation is not implemented and throws RuntimeException.","commonSituations":"S3/GCS throttling or 503s during heavy expiration of many snapshots; IAM credentials lacking s3:DeleteObjects; a custom FileIO whose bulkDelete() throws UnsupportedOperationException.","solutions":["Check the logged cause 'e' in the WARN stack trace to identify the underlying storage error (permissions, throttling, unsupported operation).","Ensure the FileIO credentials/permissions allow bulk deletion of the given file type (manifests, data, manifests-lists).","If the store throttles, reduce delete concurrency or batch size, or retry the expire operation — remaining files can be cleaned by a later run.","If the FileIO does not support bulk delete, implement bulkDelete() or configure a FileIO that does, so the non-bulk fallback path is not needed."],"exampleFix":"// before: custom FileIO without bulk delete support\npublic void deleteFiles(Iterable<String> paths) { throw new UnsupportedOperationException(); }\n// after: implement bulk delete or return unimplemented so the fallback single-delete path is used\npublic void deleteFiles(Iterable<String> pathsToRemove) {\n  Tasks.foreach(pathsToRemove).suppressFailureWhenFinished().run(io::deleteFile);\n}","handlingStrategy":"retry","validationCode":"// before running cleanup, verify delete permission on one sample path\nio.deleteFile(samplePath); // throws early if credentials are insufficient","typeGuard":null,"tryCatchPattern":"try {\n  table.expireSnapshots().expireOlderThan(ts).execute();\n} catch (RuntimeException e) {\n  LOG.warn(\"Expiration reported partial failures; will re-run for orphans\", e);\n}","preventionTips":["Grant bulk-delete permissions (e.g. s3:DeleteObjects) to the maintenance job's credentials.","Implement bulkDelete() in custom FileIO implementations.","Avoid overlapping expiration jobs; schedule a single maintenance job.","Re-run expiration periodically to clean files missed by transient failures."],"tags":["io","object-store","cleanup","deletion"],"backgroundTag":"network-request-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}