{"record":{"id":"cc47372483c8f26c","repo":"apache/seatunnel","slug":"failed-to-clean-empty-transaction-parent-directory","errorCode":null,"errorMessage":"Failed to clean empty transaction parent directory: {}","messagePattern":"Failed to clean empty transaction parent directory: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/commit/FileSinkAggregatedCommitter.java","lineNumber":209,"sourceCode":"        if (jobDir == null) {\n            return;\n        }\n        cleanupEmptyDirectory(uuidDir.toString());\n        cleanupEmptyDirectory(jobDir.toString());\n    }\n\n    private void cleanupPendingTransactionParentDirectories() {\n        pendingUuidDirectories.forEach(this::cleanupEmptyDirectory);\n        pendingJobDirectories.forEach(this::cleanupEmptyDirectory);\n        pendingUuidDirectories.clear();\n        pendingJobDirectories.clear();\n    }\n\n    private void cleanupEmptyDirectory(String directory) {\n        try {\n            hadoopFileSystemProxy.deleteEmptyDirectory(directory);\n        } catch (IOException e) {\n            log.warn(\"Failed to clean empty transaction parent directory: {}\", directory, e);\n        }\n    }\n}\n","sourceCodeStart":191,"sourceCodeEnd":213,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/sink/commit/FileSinkAggregatedCommitter.java#L191-L213","documentation":"A WARN from FileSinkAggregatedCommitter.cleanupEmptyDirectory: deleting a now-empty per-transaction parent directory (created for staging transaction files) failed with an IOException after commit. Committed data files are safe; only an empty directory may linger in the output path.","triggerScenarios":"cleanupTransactionParentDirectories -> cleanupEmptyDirectory calls hadoopFileSystemProxy.deleteEmptyDirectory(dir); the filesystem throws IOException due to permission issues, transient NameNode/S3 errors, or the directory not actually being empty (orphaned files from a failed earlier transaction).","commonSituations":"S3 eventual consistency / throttling during commit; HDFS permission mismatch between the committer user and directory owner; a crashed prior job left files in the transaction directory so it is not empty.","solutions":["Check write permissions on the output path for the job's commit user","List the directory — if non-empty, clean orphaned files from failed transactions manually","Retry after transient storage errors; safe to delete manually once all files are committed","On S3, verify bucket throttling (429s) is not interrupting cleanup"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// pre-commit check: ensure commit user can write the output tree\n// hadoop fs -test -w /output/parent && echo writable","typeGuard":null,"tryCatchPattern":"// safe to ignore for committed data; cleanup pattern:\n// try { fs.deleteEmptyDirectory(dir); } catch (IOException e) { scheduleManualCleanup(dir); }","preventionTips":["Grant the sink user recursive write/delete on the output path","Clean orphaned transaction dirs from crashed runs before re-running","Avoid heavy S3 parallelism that triggers throttled deletes during commit"],"tags":["file","hadoop","cleanup"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}