{"record":{"id":"c461cce4b635f1fb","repo":"apache/seatunnel","slug":"cannot-rename-file-from-oldpath-to-newpath","errorCode":null,"errorMessage":"Cannot rename file from [<oldPath>] to [<newPath>]: both source and target are missing.","messagePattern":"Cannot rename file from \\[<oldPath>\\] to \\[<newPath>\\]: both source and target are missing\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/hadoop/HadoopFileSystemProxy.java","lineNumber":153,"sourceCode":"            @NonNull String oldFilePath,\n            @NonNull String newFilePath,\n            boolean removeWhenNewFilePathExist)\n            throws IOException {\n        execute(\n                () -> {\n                    Path oldPath = new Path(oldFilePath);\n                    Path newPath = new Path(newFilePath);\n\n                    if (!fileExist(oldPath.toString())) {\n                        if (fileExist(newPath.toString())) {\n                            log.info(\n                                    \"Rename file from [{}] to [{}] already finished in a previous \"\n                                            + \"commit, skip.\",\n                                    oldPath,\n                                    newPath);\n                            return Void.class;\n                        }\n                        throw new IOException(\n                                \"Cannot rename file from [\"\n                                        + oldPath\n                                        + \"] to [\"\n                                        + newPath\n                                        + \"]: both source and target are missing.\");\n                    }\n\n                    if (removeWhenNewFilePathExist) {\n                        if (fileExist(newFilePath)) {\n                            getFileSystem().delete(newPath, true);\n                            log.info(\"Delete already file: {}\", newPath);\n                        }\n                    }\n                    if (!fileExist(newPath.getParent().toString())) {\n                        createDir(newPath.getParent().toString());\n                    }\n\n                    if (getFileSystem().rename(oldPath, newPath)) {","sourceCodeStart":135,"sourceCodeEnd":171,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/hadoop/HadoopFileSystemProxy.java#L135-L171","documentation":"HadoopFileSystemProxy.renameFile renames the sink's in-progress file to its final committed name. When neither the source nor the target path exists on the filesystem, it cannot treat the rename as already-done or as a no-op, so it throws an IOException describing both paths.","triggerScenarios":"Commit phase calls renameFile(oldPath, newPath) after the source temp file was deleted (e.g. manually cleaned, expired storage lifecycle rule, or a different cluster/filesystem) and the target was never created — the 'both missing' branch of the existence check.","commonSituations":"Retrying a commit against a storage where temp files were garbage-collected; pointing sink and recovery state at different HDFS/S3 clusters; manually removing _transaction files between failed and retried jobs; misconfigured checkpoint storage so fileStates reference stale paths.","solutions":["Verify the source temp file exists on the configured filesystem before retrying the commit","Ensure the sink filesystem (fs.defaultFS / path scheme) matches the one used when the files were written","Restore the missing file from backup or rerun the job from an earlier checkpoint so temp files are regenerated","Check storage lifecycle/expiry policies (e.g. S3 lifecycle rules) that may delete temp files"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"FileSystem fs = FileSystem.get(conf);\nif (!fs.exists(oldPath) && !fs.exists(newPath)) {\n    throw new IllegalStateException(\"Rename source and target both missing: \" + oldPath + \" -> \" + newPath);\n}","typeGuard":null,"tryCatchPattern":"try {\n    proxy.renameFile(oldPath, newPath);\n} catch (IOException e) {\n    if (e.getMessage().contains(\"both source and target are missing\")) {\n        // verify filesystem config / restore temp files before retrying commit\n    } else {\n        throw e;\n    }\n}","preventionTips":["Keep sink filesystem config identical across retries (same defaultFS/scheme)","Disable storage lifecycle rules that could delete temp/transaction directories","Do not manually clean transaction temp files between failed and retried runs"],"tags":["hadoop","file-rename","commit"],"backgroundTag":"file-not-found","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}