{"record":{"id":"2aff9abd821c4156","repo":"apache/seatunnel","slug":"failed-to-rename-tmp-file-to-final-file","errorCode":null,"errorMessage":"Failed to rename tmp file to final file","messagePattern":"Failed to rename tmp file to final file","errorType":"exception","errorClass":"CheckpointStorageException","httpStatus":null,"severity":"critical","filePath":"seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/HdfsStorage.java","lineNumber":120,"sourceCode":"                new Path(\n                        getStorageParentDirectory()\n                                + state.getJobId()\n                                + \"/\"\n                                + getCheckPointName(state)\n                                + STORAGE_TMP_SUFFIX);\n        try (FSDataOutputStream out = fs.create(tmpFilePath, false)) {\n            out.write(datas);\n        } catch (IOException e) {\n            throw new CheckpointStorageException(\n                    String.format(\n                            \"Failed to write checkpoint data, file: %s, state: %s\",\n                            tmpFilePath, state),\n                    e);\n        }\n        try {\n            boolean success = fs.rename(tmpFilePath, filePath);\n            if (!success) {\n                throw new CheckpointStorageException(\"Failed to rename tmp file to final file\");\n            }\n\n        } catch (IOException e) {\n            throw new CheckpointStorageException(\"Failed to rename tmp file to final file\");\n        } finally {\n            try {\n                // clean up tmp file, if still lying around\n                if (fs.exists(tmpFilePath)) {\n                    fs.delete(tmpFilePath, false);\n                }\n            } catch (IOException ioe) {\n                log.error(\"Failed to delete tmp file\", ioe);\n            }\n        }\n\n        return filePath.getName();\n    }\n","sourceCodeStart":102,"sourceCodeEnd":138,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/HdfsStorage.java#L102-L138","documentation":"After writing the tmp checkpoint file, storeCheckPoint renames it to the final path. If fs.rename returns false (no exception but rename not performed) the method throws this CheckpointStorageException. The tmp file is then deleted in the finally block. This is the 'silent failure' variant of the rename error, where Hadoop simply reports the rename did not happen.","triggerScenarios":"storeCheckPoint (also via modifyResumeTokenInCheckpoint) when fs.rename(tmpFilePath, filePath) returns false — typically because the target parent directory does not exist, the target already exists and is a directory, or the source tmp file vanished (deleted by another process/concurrent writer).","commonSituations":"Checkpoint parent directory removed or never created on HDFS, two writers with the same jobId racing, tmp file already cleaned by a retention/cleanup job, or HDFS permission allowing write of new files but not rename operations in the target dir.","solutions":["Verify the checkpoint parent directory exists and is writable (hdfs dfs -mkdir -p; hdfs dfs -chmod)","Check the nested finally-cleanup logs to confirm the tmp file existed before rename","Prevent duplicate jobIds / concurrent writers racing on the same checkpoint path","Check HDFS permissions on both source and target paths (rename needs write on both dirs)","Inspect NameNode logs around the failure time for the specific rename denial reason"],"exampleFix":"// before\n// assume parent dir exists\n// after\nif (!fs.exists(parentDir)) { fs.mkdirs(parentDir); }\nboolean success = fs.rename(tmpFilePath, filePath);","handlingStrategy":"try-catch","validationCode":"// pre-flight: ensure destination parent exists and is writable\nif (!fs.exists(parentDir)) fs.mkdirs(parentDir);\nFsPermission perm = fs.getFileStatus(parentDir).getPermission();\n// verify action write access for the submitting user","typeGuard":null,"tryCatchPattern":"try {\n  storage.storeCheckPoint(state);\n} catch (CheckpointStorageException e) {\n  if (e.getMessage().equals(\"Failed to rename tmp file to final file\")) {\n    log.error(\"Rename returned false: check parent dir existence, permissions, and concurrent writers\");\n  } else throw e;\n}","preventionTips":["Create the checkpoint parent directory before job submission and never delete it while jobs run","Ensure only one writer uses a given jobId/checkpoint path","Verify write permission on BOTH source and destination directories (rename requires both)","Check NameNode logs when rename silently fails — the exception carries no cause"],"tags":["hdfs","checkpoint-storage","rename"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}