{"record":{"id":"04b49e06d04901bf","repo":"apache/iceberg","slug":"deleted-only-of-files-using-bulk-deletes-04b49e","errorCode":null,"errorMessage":"Deleted only {} of {} files using bulk deletes","messagePattern":"Deleted only (.+?) of (.+?) files using bulk deletes","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java","lineNumber":325,"sourceCode":"  }\n\n  private void collectPathsForOutput(\n      List<String> paths, List<String> orphanFileList, int maxSampleSize) {\n    if (streamResults()) {\n      int lengthToAdd = Math.min(maxSampleSize - orphanFileList.size(), paths.size());\n      orphanFileList.addAll(paths.subList(0, lengthToAdd));\n    } else {\n      orphanFileList.addAll(paths);\n    }\n  }\n\n  private void deleteBulk(SupportsBulkOperations io, List<String> paths) {\n    try {\n      io.deleteFiles(paths);\n      LOG.info(\"Deleted {} files using bulk deletes\", paths.size());\n    } catch (BulkDeletionFailureException e) {\n      int deletedFilesCount = paths.size() - e.numberFailedObjects();\n      LOG.warn(\n          \"Deleted only {} of {} files using bulk deletes\", deletedFilesCount, paths.size(), e);\n    }\n  }\n\n  private void deleteNonBulk(List<String> paths) {\n    Tasks.Builder<String> deleteTasks =\n        Tasks.foreach(paths)\n            .noRetry()\n            .executeWith(deleteExecutorService)\n            .suppressFailureWhenFinished()\n            .onFailure((file, exc) -> LOG.warn(\"Failed to delete file: {}\", file, exc));\n\n    if (deleteFunc == null) {\n      LOG.info(\n          \"Table IO {} does not support bulk operations. Using non-bulk deletes.\",\n          table.io().getClass().getName());\n      deleteTasks.run(table.io()::deleteFile);\n    } else {","sourceCodeStart":307,"sourceCodeEnd":343,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java#L307-L343","documentation":"This is a WARN log, not a thrown exception, emitted when a bulk delete against a SupportsBulkOperations FileIO partially fails. The FileIO raised BulkDeletionFailureException carrying numberFailedObjects(); the action computes how many paths were actually deleted and logs the shortfall along with the exception for the stack trace.","triggerScenarios":"DeleteOrphanFilesSparkAction.deleteFiles dispatches to deleteBulk because table.io() implements SupportsBulkOperations, and io.deleteFiles(paths) throws BulkDeletionFailureException (e.g. S3 multipart delete returns individual object failures).","commonSituations":"S3ObjectStoreFileIO/HadoopFileIO bulk deletes where some objects were already gone, are write-protected, or hit transient S3 errors; stale orphan-file paths that another concurrent expiration already removed.","solutions":["Inspect the attached exception stack for the underlying per-object delete errors and fix those (permissions, lifecycle conflicts).","Re-run the RemoveOrphanFiles action; the remaining undeleted files are retried and the operation is idempotent.","If failures are because files were concurrently deleted, ignore the warning — partial success is expected in that race.","If the FileIO's bulk-delete endpoint is flaky, force the non-bulk path by using a FileIO that does not implement SupportsBulkOperations or a custom deleteFunc via deleteWith."],"exampleFix":"// before: assuming all files deleted\nio.deleteFiles(paths);\n// after: treat partial failure as recoverable\ntry {\n  io.deleteFiles(paths);\n} catch (BulkDeletionFailureException e) {\n  List<String> retry = ...; // re-run deleteFile on failed paths individually\n}","handlingStrategy":"retry","validationCode":"boolean bulk = table.io() instanceof org.apache.iceberg.io.SupportsBulkOperations;","typeGuard":"if (table.io() instanceof SupportsBulkOperations io) { /* bulk path */ }","tryCatchPattern":"try { io.deleteFiles(paths); } catch (BulkDeletionFailureException e) { List<String> remaining = /* paths minus e.deletedCount */; retryDeleteIndividually(remaining); }","preventionTips":["Treat bulk deletes as best-effort; always plan a re-run for stragglers","Check FileIO permissions and lifecycle policies before large orphans cleanup","Monitor BulkDeletionFailureException counts as an ops signal"],"tags":["logging","spark","file-deletion","partial-failure"],"backgroundTag":"partial-bulk-delete-failure","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}