Scheduler timed out during close; relying on local buffer as

Error message

Scheduler timed out during close; relying on local buffer as fallback. Could not determine dropped event count: {}

What it means

A companion warning to the unsynced-count warning: when computing how many ring buffer events were skipped during a timed-out close(), an exception occurred (e.g. HazelcastInstanceNotActiveException because the ring buffer is already gone), so the handler logs that it cannot determine the dropped count and will rely on the local buffer fallback.

Source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/event/JobEventLocalFileHandler.java:430

            try {
                finallyTerminated = scheduledExecutorService.awaitTermination(2, TimeUnit.SECONDS);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
            // Log how many ringbuffer events may have been skipped for observability.
            try {
                long tail = ringbuffer.tailSequence();
                long unsynced = tail - committedEventIndex + 1;
                if (unsynced > 0) {
                    log.warn(
                            "Scheduler timed out during close; up to {} ringbuffer event(s) were"
                                    + " not flushed to disk. Local buffer (cap={}) will be drained"
                                    + " as fallback.",
                            unsynced,
                            LOCAL_EVENT_BUFFER_CAPACITY);
                }
            } catch (Exception e) {
                log.warn(
                        "Scheduler timed out during close; relying on local buffer as fallback."
                                + " Could not determine dropped event count: {}",
                        e.getMessage());
            }
            if (!finallyTerminated) {
                // The scheduler thread is still alive even after shutdownNow().  Draining
                // the local buffer here would race with the scheduler on writerLock.
                // closing=true already prevents the scheduler from opening new files,
                // so we can safely close the current writer under the lock and return.
                log.warn(
                        "Scheduler thread did not terminate after shutdownNow();"
                                + " skipping local buffer drain to avoid concurrent writer access.");
                synchronized (writerLock) {
                    TraceFileWriter writer = currentWriter;
                    currentWriter = null;
                    if (writer != null) {
                        try {
                            writer.close();

View on GitHub (pinned to cf67b549a7)

Solutions

Accept — this is a best-effort observability path; the local buffer drain is the fallback.
Close the JobEventLocalFileHandler before tearing down the Hazelcast instance to preserve accurate flush accounting.
Check shutdown hooks ordering in custom launchers that may close resources out of order.

Example fix

null

Defensive patterns

Strategy: try-catch

Try / catch

try { long tail = ringbuffer.tailSequence(); } catch (HazelcastInstanceNotActiveException e) { /* instance gone; rely on local buffer */ }

Prevention

Close event handlers before destroying the Hazelcast instance.
Treat observability accounting as best-effort during shutdown.

When it happens

Trigger: close() times out waiting for the scheduler, then tailSequence() or committedEventIndex access throws — typically because Hazelcast is shutting down and the ring buffer proxy is no longer valid.

Common situations: Cluster shutdown ordering where the event handler closes after the Hazelcast instance is destroyed.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/9ddc7350c9da5625. Report an issue: GitHub.