{"record":{"id":"5d95b7b17eacc45f","repo":"apache/seatunnel","slug":"skipping-deletion-log-directory-either-does","errorCode":null,"errorMessage":"Skipping deletion: Log directory '{}' either does not exist or is not a valid directory. Please verify the path and ensure the logs are being written correctly.","messagePattern":"Skipping deletion: Log directory '(.+?)' either does not exist or is not a valid directory\\. Please verify the path and ensure the logs are being written correctly\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/telemetry/log/TaskLogManagerService.java","lineNumber":73,"sourceCode":"        if (path == null || jobId <= 0) {\n            return;\n        }\n        String[] logFiles = getLogFiles(jobId, path);\n        for (String logFile : logFiles) {\n            String logPath = path + \"/\" + logFile;\n            log.info(\"Cleaning logs for jobId: {} , path : {}\", jobId, logPath);\n            try {\n                Files.delete(Paths.get(logPath));\n            } catch (IOException e) {\n                log.warn(\"Failed to delete log file: {}\", logPath, e);\n            }\n        }\n    }\n\n    private String[] getLogFiles(long jobId, String path) {\n        File logDir = new File(path);\n        if (!logDir.exists() || !logDir.isDirectory()) {\n            log.warn(\n                    \"Skipping deletion: Log directory '{}' either does not exist or is not a valid directory. Please verify the path and ensure the logs are being written correctly.\",\n                    path);\n            return new String[0];\n        }\n\n        return logDir.list((dir, name) -> name.contains(String.valueOf(jobId)));\n    }\n}\n","sourceCodeStart":55,"sourceCodeEnd":82,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/telemetry/log/TaskLogManagerService.java#L55-L82","documentation":"TaskLogManagerService.getLogFiles validates that the configured log directory exists and is a directory before listing job logs. If not, it warns and returns an empty array so no deletion is attempted. This means log cleanup silently skips for a bad/missing path, so old logs accumulate elsewhere or the path is misconfigured.","triggerScenarios":"clean() calls getLogFiles(jobId, path) where 'path' does not exist or is a file, not a directory — typically a wrong log-dire path in telemetry/task-log-manager configuration or a path deleted externally.","commonSituations":"Typo in log path config; logs moved to another directory after an upgrade; directory wiped by tmp-cleaners (path under /tmp); running cleanup on a node that never writes logs to that path.","solutions":["Verify the configured log directory path exists on every cluster node and correct the task-log-manager configuration.","Create the directory (mkdir -p) with proper ownership for the SeaTunnel user.","Check if an external cleanup (cron/tmpfiles) removes the dir and exclude it.","Confirm each worker writes logs to the same configured path to avoid node-specific misses."],"exampleFix":"// before: config points to nonexistent dir\nlog-dir: /data/seatunnel/job-logs   # directory was never created\n// after\nmkdir -p /data/seatunnel/job-logs && chown seatunnel:seatunnel /data/seatunnel/job-logs\nlog-dir: /data/seatunnel/job-logs","handlingStrategy":"validation","validationCode":"// validate log dir before configuring cleanup\nPath dir = Paths.get(logDir);\nif (!Files.isDirectory(dir)) {\n    throw new IllegalArgumentException(\"log-dir does not exist or is not a directory: \" + logDir);\n}","typeGuard":"boolean isValidLogDir(String p) {\n    File d = new File(p);\n    return d.exists() && d.isDirectory();\n}","tryCatchPattern":null,"preventionTips":["Verify the configured log directory exists on every cluster node at startup.","Create and own the directory as the SeaTunnel user before enabling log cleanup.","Exclude the log dir from OS tmp cleaners (tmpwatch/systemd-tmpfiles).","Re-check the path after upgrades that relocate logs."],"tags":["logs","configuration","path-validation","cleanup"],"backgroundTag":"directory-not-found","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}