{"record":{"id":"5f5940e8b9a09388","repo":"apache/seatunnel","slug":"delete-transaction-directory-failed-after-suc","errorCode":null,"errorMessage":"delete transaction directory [{}] failed after successful commit, ignore this cleanup error.","messagePattern":"delete transaction directory \\[(.+?)\\] failed after successful commit, ignore this cleanup error\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/commit/FileSinkAggregatedCommitter.java","lineNumber":75,"sourceCode":"        aggregatedCommitInfos.forEach(\n                aggregatedCommitInfo -> {\n                    try {\n                        for (Map.Entry<String, LinkedHashMap<String, String>> entry :\n                                aggregatedCommitInfo.getTransactionMap().entrySet()) {\n                            for (Map.Entry<String, String> mvFileEntry :\n                                    entry.getValue().entrySet()) {\n                                // first rename temp file\n                                hadoopFileSystemProxy.renameFile(\n                                        mvFileEntry.getKey(), mvFileEntry.getValue(), true);\n                            }\n                            String transactionDir = entry.getKey();\n                            // Data files are already committed after rename; tmp cleanup is\n                            // best-effort and should not fail the whole checkpoint.\n                            try {\n                                hadoopFileSystemProxy.deleteFile(transactionDir);\n                                registerTransactionParentDirectories(transactionDir);\n                            } catch (Exception cleanupException) {\n                                log.warn(\n                                        \"delete transaction directory [{}] failed after successful commit, ignore this cleanup error.\",\n                                        transactionDir,\n                                        cleanupException);\n                            }\n                        }\n                    } catch (Throwable e) {\n                        log.error(\n                                \"commit aggregatedCommitInfo error, aggregatedCommitInfo = {} \",\n                                aggregatedCommitInfo,\n                                e);\n                        errorAggregatedCommitInfoList.add(aggregatedCommitInfo);\n                    }\n                });\n        return errorAggregatedCommitInfoList;\n    }\n\n    /**\n     * The logic about how to combine commit message.","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/commit/FileSinkAggregatedCommitter.java#L57-L93","documentation":"FileSinkAggregatedCommitter.commit, after successfully committing (renaming) files, deletes the temporary transaction directory as a best-effort cleanup. If hadoopFileSystemProxy.deleteFile throws, the error is logged with this message and deliberately NOT propagated, because the data files are already committed and cleanup failure must not fail the checkpoint.","triggerScenarios":"deleteFile on the transaction directory throws (permissions on the parent dir, directory already removed by another committer/retry, HDFS/S3 transient error) after a successful rename-based commit.","commonSituations":"Object stores (S3/OSS) with eventual consistency where a just-renamed path briefly appears still present; concurrent committer retries racing on the same transaction dir; HDFS permission changes on staging directories.","solutions":["Safe to ignore for data correctness — files are already committed; just remove leftover tmp dirs manually","Check filesystem permissions on the staging/transaction parent directories","Ensure only one committer instance handles a given transaction (avoid duplicate commits from retries)","Configure lifecycle/TTL cleanup on the staging directory if leftovers accumulate"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Before commit, verify staging dir state\nif (!hadoopFileSystemProxy.fileExist(transactionDir)) { return; } // nothing to clean up","typeGuard":null,"tryCatchPattern":"try { committer.commit(committables); } catch (Exception e) { /* real commit failures surface here; cleanup warnings never do — check logs for 'ignore this cleanup error' */ throw e; }","preventionTips":["Ensure staging/transaction parent dirs are writable by the job user","Avoid concurrent committers on the same transaction dir","On object stores, expect eventual-consistency cleanup warnings and schedule TTL cleanup"],"tags":["file","committer","cleanup","hadoop"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}