{"record":{"id":"9d07933ef30cc0fa","repo":"apache/seatunnel","slug":"failed-to-write-checkpoint-data-file-tmpfilepa","errorCode":null,"errorMessage":"Failed to write checkpoint data, file: ${tmpFilePath}, state: ${state}","messagePattern":"Failed to write checkpoint data, file: (.+?), state: (.+?)","errorType":"exception","errorClass":"CheckpointStorageException","httpStatus":null,"severity":"critical","filePath":"seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/HdfsStorage.java","lineNumber":111,"sourceCode":"        }\n        Path filePath =\n                new Path(\n                        getStorageParentDirectory()\n                                + state.getJobId()\n                                + \"/\"\n                                + getCheckPointName(state));\n\n        Path tmpFilePath =\n                new Path(\n                        getStorageParentDirectory()\n                                + state.getJobId()\n                                + \"/\"\n                                + getCheckPointName(state)\n                                + STORAGE_TMP_SUFFIX);\n        try (FSDataOutputStream out = fs.create(tmpFilePath, false)) {\n            out.write(datas);\n        } catch (IOException e) {\n            throw new CheckpointStorageException(\n                    String.format(\n                            \"Failed to write checkpoint data, file: %s, state: %s\",\n                            tmpFilePath, state),\n                    e);\n        }\n        try {\n            boolean success = fs.rename(tmpFilePath, filePath);\n            if (!success) {\n                throw new CheckpointStorageException(\"Failed to rename tmp file to final file\");\n            }\n\n        } catch (IOException e) {\n            throw new CheckpointStorageException(\"Failed to rename tmp file to final file\");\n        } finally {\n            try {\n                // clean up tmp file, if still lying around\n                if (fs.exists(tmpFilePath)) {\n                    fs.delete(tmpFilePath, false);","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-hdfs/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/hdfs/HdfsStorage.java#L93-L129","documentation":"After serializing, storeCheckPoint writes the bytes to a temporary HDFS file (final name + tmp suffix) via fs.create/FSDataOutputStream. An IOException during create or write is wrapped in a CheckpointStorageException identifying the temp file path and the state, so the developer can see exactly which file failed to write.","triggerScenarios":"storeCheckPoint (also via modifyResumeTokenInCheckpoint) when fs.create(tmpFilePath, false) or out.write(datas) throws IOException: HDFS disk full, DataNode unavailable, file already exists with create(false), lease recovery conflicts, or NameNode in safe mode.","commonSituations":"HDFS cluster at capacity (disk or inodes), flaky network to DataNodes, concurrent writers colliding on the same tmp file name for the same job/pipeline, HDFS in safe mode after restart, permission problems on the checkpoint directory.","solutions":["Check HDFS health: hdfs dfsadmin -report and dfsadmin -safemode get; clear safe mode or wait","Free HDFS space or expand the cluster if disk/inode usage is near capacity","Verify write permissions for the SeaTunnel user on the checkpoint parent directory","Look for leftover .tmp files and stale writers; avoid running duplicate jobs with the same jobId","Inspect the nested IOException cause for the specific DataNode/NameNode error"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// pre-flight: writable target dir and adequate space\nif (!fs.exists(parentDir)) fs.mkdirs(parentDir);\n// check quota/space via hdfs dfsadmin -report / dfs quota commands","typeGuard":null,"tryCatchPattern":"try {\n  storage.storeCheckPoint(state);\n} catch (CheckpointStorageException e) {\n  if (e.getMessage().startsWith(\"Failed to write checkpoint data\")) {\n    log.error(\"HDFS write failed for \" + tmpFilePath + \", retrying after health check\", e);\n    // check safemode/disk space, then retry with backoff\n  } else throw e;\n}","preventionTips":["Monitor HDFS disk/inode usage and directory quotas","Ensure the SeaTunnel user has write permission on the checkpoint directory","Avoid duplicate jobs sharing the same jobId and checkpoint paths","Keep the cluster out of safe mode; alert on DataNode loss"],"tags":["hdfs","checkpoint-storage","file-write"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}