{"record":{"id":"0012106285e48285","repo":"apache/seatunnel","slug":"scheduler-thread-did-not-terminate-after-shutdownn","errorCode":null,"errorMessage":"Scheduler thread did not terminate after shutdownNow(); skipping local buffer drain to avoid concurrent writer access.","messagePattern":"Scheduler thread did not terminate after shutdownNow\\(\\); skipping local buffer drain to avoid concurrent writer access\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/event/JobEventLocalFileHandler.java","lineNumber":440,"sourceCode":"                    log.warn(\n                            \"Scheduler timed out during close; up to {} ringbuffer event(s) were\"\n                                    + \" not flushed to disk. Local buffer (cap={}) will be drained\"\n                                    + \" as fallback.\",\n                            unsynced,\n                            LOCAL_EVENT_BUFFER_CAPACITY);\n                }\n            } catch (Exception e) {\n                log.warn(\n                        \"Scheduler timed out during close; relying on local buffer as fallback.\"\n                                + \" Could not determine dropped event count: {}\",\n                        e.getMessage());\n            }\n            if (!finallyTerminated) {\n                // The scheduler thread is still alive even after shutdownNow().  Draining\n                // the local buffer here would race with the scheduler on writerLock.\n                // closing=true already prevents the scheduler from opening new files,\n                // so we can safely close the current writer under the lock and return.\n                log.warn(\n                        \"Scheduler thread did not terminate after shutdownNow();\"\n                                + \" skipping local buffer drain to avoid concurrent writer access.\");\n                synchronized (writerLock) {\n                    TraceFileWriter writer = currentWriter;\n                    currentWriter = null;\n                    if (writer != null) {\n                        try {\n                            writer.close();\n                        } catch (IOException e) {\n                            log.error(\"Failed to close current writer\", e);\n                        }\n                    }\n                }\n                return;\n            }\n            try {\n                reportFromRingbuffer(true);\n            } catch (HazelcastInstanceNotActiveException e) {","sourceCodeStart":422,"sourceCodeEnd":458,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/event/JobEventLocalFileHandler.java#L422-L458","documentation":"If the scheduler thread is still alive even after shutdownNow(), close() skips draining the local buffer because doing so would race with the scheduler on writerLock. It logs this warning and instead safely closes the current trace file writer under the lock.","triggerScenarios":"JobEventLocalFileHandler.close() awaited scheduler termination, timed out, then called shutdownNow(), and finallyTerminated is still false — a stuck or blocked scheduler thread (e.g. blocked on file I/O or an internal lock).","commonSituations":"Scheduler thread blocked on a hung file system (NFS, full disk) or a deadlock during cluster shutdown; events still in the local buffer are intentionally not written to avoid corruption.","solutions":["Diagnose why the scheduler thread is blocked — take a thread dump (jstack) at shutdown to find the stuck frame.","Check for filesystem hangs (NFS timeouts, disk full, permission issues) on the trace output directory.","Pending local-buffer events are dropped in this path; reduce event volume or fix I/O to avoid hitting it."],"exampleFix":"null","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { handler.close(); } catch (Exception e) { log.warn(\"scheduler thread stuck; local buffer drain skipped\", e); }","preventionTips":["Capture thread dumps on slow shutdowns to find blocked scheduler threads.","Avoid network filesystems for trace output.","Verify disk space and permissions on the trace directory."],"tags":["shutdown","concurrency","scheduler","observability"],"backgroundTag":"invalid-state-transition","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}