{"record":{"id":"4c486233e9056f74","repo":"apache/seatunnel","slug":"failed-to-delete-checkpoint-directory","errorCode":null,"errorMessage":"Failed to delete checkpoint directory ","messagePattern":"Failed to delete checkpoint directory ","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-local-file/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/localfile/LocalFileStorage.java","lineNumber":281,"sourceCode":"                        } catch (IOException e) {\n                            log.error(\n                                    \"Failed to read checkpoint data from file \"\n                                            + file.getAbsolutePath(),\n                                    e);\n                        }\n                    }\n                });\n        return pipelineStates;\n    }\n\n    @Override\n    public void deleteCheckpoint(String jobId) {\n        String jobPath = getStorageParentDirectory() + jobId;\n        File file = new File(jobPath);\n        try {\n            FileUtils.deleteDirectory(file);\n        } catch (IOException e) {\n            log.warn(\"Failed to delete checkpoint directory \" + jobPath, e);\n        }\n    }\n\n    @Override\n    public PipelineState getCheckpoint(String jobId, String pipelineId, String checkpointId)\n            throws CheckpointStorageException {\n        String parentPath = getStorageParentDirectory() + jobId;\n        Collection<File> fileList = new ArrayList<>();\n        try {\n            fileList = FileUtils.listFiles(new File(parentPath), FILE_EXTENSIONS, false);\n        } catch (Exception e) {\n            if (!(e.getCause() instanceof NoSuchFileException)) {\n                throw new CheckpointStorageException(ExceptionUtils.getMessage(e));\n            }\n        }\n        if (fileList.isEmpty()) {\n            log.info(\"No checkpoint found for this job,  the job id is: \" + jobId);\n            return null;","sourceCodeStart":263,"sourceCodeEnd":299,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-storage/checkpoint-storage-plugins/checkpoint-storage-local-file/src/main/java/org/apache/seatunnel/engine/checkpoint/storage/localfile/LocalFileStorage.java#L263-L299","documentation":"LocalFileStorage.deleteCheckpoint deletes the job's checkpoint directory recursively via FileUtils.deleteDirectory. On IOException the failure is only logged at WARN and swallowed, leaving orphaned checkpoint files on local disk. Storage may grow over time and job cleanup is only partially done.","triggerScenarios":"FileUtils.deleteDirectory(new File(getStorageParentDirectory()+jobId)) throws IOException — file locked/in use, permission denied, or the directory disappears mid-delete (concurrent cleanup).","commonSituations":"Checkpoint dir under /tmp cleaned by OS tmpwatch/systemd-tmpfiles during delete; disk permission issues after running SeaTunnel as different users; a running job on the same node still writing checkpoints while another process deletes them.","solutions":["Ensure the job is fully terminated before cleanup and only one cleanup path exists (avoid deleting checkpoints of a still-running job).","Check permissions/ownership of the local checkpoint storage directory for the SeaTunnel process user.","Move checkpoint storage off tmpfs/tmpwatch-managed locations or exclude it from OS cleanup.","Manually remove the orphaned jobPath directory once nothing is using it."],"exampleFix":"// before: checkpoints under /tmp removed by tmpwatch mid-delete\nstorage-dir: /tmp/seatunnel/checkpoint-storage\n// after\nstorage-dir: /var/lib/seatunnel/checkpoint-storage","handlingStrategy":"retry","validationCode":"// pre-check local checkpoint dir is deletable\nFile dir = new File(storageRoot, jobId);\nif (dir.exists() && !dir.canWrite()) {\n    log.warn(\"cannot delete checkpoint dir {}\", dir);\n}","typeGuard":"boolean isDeletableDir(File d) {\n    return d.isDirectory() && d.canWrite();\n}","tryCatchPattern":"try {\n    localFileStorage.deleteCheckpoint(jobId);\n} catch (Throwable t) {\n    // engine swallows IOException; orphaned dir may remain\n    scheduleManualCleanup(new File(storageRoot, jobId));\n}","preventionTips":["Store checkpoints outside /tmp or tmpwatch-managed paths.","Ensure the job is terminated before deleting its checkpoints.","Run SeaTunnel nodes under a consistent service user with write access to the storage dir.","Periodically sweep the storage parent dir for orphaned job directories."],"tags":["local-file","checkpoint","cleanup","io"],"backgroundTag":"file-delete-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}