{"record":{"id":"9743576840803453","repo":"apache/iceberg","slug":"failed-to-delete-file-974357","errorCode":null,"errorMessage":"Failed to delete file: {}","messagePattern":"Failed to delete file: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java","lineNumber":336,"sourceCode":"\n  private void deleteBulk(SupportsBulkOperations io, List<String> paths) {\n    try {\n      io.deleteFiles(paths);\n      LOG.info(\"Deleted {} files using bulk deletes\", paths.size());\n    } catch (BulkDeletionFailureException e) {\n      int deletedFilesCount = paths.size() - e.numberFailedObjects();\n      LOG.warn(\n          \"Deleted only {} of {} files using bulk deletes\", deletedFilesCount, paths.size(), e);\n    }\n  }\n\n  private void deleteNonBulk(List<String> paths) {\n    Tasks.Builder<String> deleteTasks =\n        Tasks.foreach(paths)\n            .noRetry()\n            .executeWith(deleteExecutorService)\n            .suppressFailureWhenFinished()\n            .onFailure((file, exc) -> LOG.warn(\"Failed to delete file: {}\", file, exc));\n\n    if (deleteFunc == null) {\n      LOG.info(\n          \"Table IO {} does not support bulk operations. Using non-bulk deletes.\",\n          table.io().getClass().getName());\n      deleteTasks.run(table.io()::deleteFile);\n    } else {\n      LOG.info(\"Custom delete function provided. Using non-bulk deletes\");\n      deleteTasks.run(deleteFunc::accept);\n    }\n  }\n\n  @VisibleForTesting\n  static Dataset<String> findOrphanFiles(\n      Dataset<FileURI> actualFileIdentDS,\n      Dataset<FileURI> validFileIdentDS,\n      PrefixMismatchMode prefixMismatchMode) {\n","sourceCodeStart":318,"sourceCodeEnd":354,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/DeleteOrphanFilesSparkAction.java#L318-L354","documentation":"WARN log produced by the Tasks.foreach onFailure callback in deleteNonBulk. When the table's FileIO lacks bulk-delete support, each path is deleted one-by-one via table.io().deleteFile on an executor; any per-file exception (with no retries configured) is logged here instead of failing the job.","triggerScenarios":"DeleteOrphanFilesSparkAction.deleteFiles with a FileIO that does not implement SupportsBulkOperations (or an explicit deleteFunc), and io.deleteFile(path) throws for a specific file — permissions, NoSuchKey races, network errors.","commonSituations":"Local/Hadoop file systems or custom FileIOs without bulk support; files already removed by a concurrent job; permission or lifecycle-policy blocks on individual objects.","solutions":["Read the logged exception to identify the failing file and root cause (ACLs, absence, throttling).","Re-run ExpireSnapshots/RemoveOrphanFiles — deleting an already-missing file is expected to fail harmlessly and remaining files are retried.","Fix filesystem permissions or object-lock/lifecycle settings for the affected paths.","Switch to a FileIO supporting bulk deletes (e.g. S3FileIO) so deletes go through the bulk path with better parallelism."],"exampleFix":"// before: silent assumption deletes succeed\ntasks.run(table.io()::deleteFile);\n// after: optionally supply a tolerant delete function\naction.deleteWith(path -> {\n  try { io.deleteFile(path); } catch (Exception e) { LOG.warn(\"skip {}: {}\", path, e.toString()); }\n});","handlingStrategy":"retry","validationCode":"// ensure delete permissions before the action\ntable.io().deleteFile(testPath);","typeGuard":"if (!(table.io() instanceof SupportsBulkOperations)) { /* non-bulk path: expect per-file warnings */ }","tryCatchPattern":"try { io.deleteFile(path); } catch (Exception e) { LOG.warn(\"Failed to delete file: {}\", path, e); failedPaths.add(path); }","preventionTips":["Pre-verify write/delete permissions on the orphan-file location","Expect and tolerate failures for files concurrently removed by other jobs","Supply a tolerant deleteWith callback if missing files are normal"],"tags":["logging","spark","file-deletion"],"backgroundTag":"partial-bulk-delete-failure","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}