{"record":{"id":"50ed426c9a7a7270","repo":"apache/iceberg","slug":"deleted-only-of-file-s-50ed42","errorCode":null,"errorMessage":"Deleted only {} of {} file(s) ({})","messagePattern":"Deleted only (.+?) of (.+?) file\\(s\\) \\((.+?)\\)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkCleanupUtil.java","lineNumber":115,"sourceCode":"    Tasks.foreach(paths)\n        .executeWith(ThreadPools.getWorkerPool())\n        .stopRetryOn(NotFoundException.class)\n        .suppressFailureWhenFinished()\n        .onFailure((path, exc) -> LOG.warn(\"Failed to delete {} ({})\", path, context, exc))\n        .retry(DELETE_NUM_RETRIES)\n        .exponentialBackoff(\n            DELETE_MIN_RETRY_WAIT_MS,\n            DELETE_MAX_RETRY_WAIT_MS,\n            DELETE_TOTAL_RETRY_TIME_MS,\n            2 /* exponential */)\n        .run(\n            path -> {\n              io.deleteFile(path);\n              deletedFilesCount.incrementAndGet();\n            });\n\n    if (deletedFilesCount.get() < paths.size()) {\n      LOG.warn(\"Deleted only {} of {} file(s) ({})\", deletedFilesCount, paths.size(), context);\n    } else {\n      LOG.info(\"Deleted {} file(s) ({})\", paths.size(), context);\n    }\n  }\n}\n","sourceCodeStart":97,"sourceCodeEnd":121,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkCleanupUtil.java#L97-L121","documentation":"A WARN log emitted by SparkCleanupUtil.delete after the Tasks.foreach loop completes when fewer files were deleted than requested (some deletions failed or files were already gone). It indicates partially successful cleanup of written/orphan data files.","triggerScenarios":"deleteFiles path (abort cleanup, deleteWhere) counts increments only after successful io.deleteFile; any suppressed failures (including stopRetryOn NotFoundException) leave deletedFilesCount below paths.size().","commonSituations":"Orphan files left behind after aborted Spark jobs when some storage operations fail; concurrent writers deleting the same files (NotFoundException).","solutions":["Check logs for the accompanying 'Failed to delete' warnings to see which paths failed and why","Run the remove_orphan_files procedure to sweep remaining orphan files","Fix underlying storage permissions or throttling issues","Treat as benign when concurrent writers intentionally removed the files"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// after aborts, verify orphan files\nCALL catalog.system.remove_orphan_files(table => 'db.t', older_than => TIMESTAMP '2026-09-10 00:00:00');","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Run remove_orphan_files on a regular maintenance schedule","Avoid concurrent writers deleting the same files","Alert on this warning to catch systematic storage problems"],"tags":["spark","cleanup","file-deletion"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}