apache/seatunnel · warning

Timed out waiting for http report scheduler to stop

Error message

Timed out waiting for http report scheduler to stop

What it means

Logged when close() on the HTTP job-event reporter could not confirm the internal report scheduler terminated within its shutdown timeout. Because the scheduler is still running, the final flush from the ring buffer is skipped to avoid concurrent access; remaining events may not be reported.

Source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/event/JobEventHttpReportHandler.java:228

            schedulerTerminated = scheduledExecutorService.awaitTermination(5, TimeUnit.SECONDS);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        if (!schedulerTerminated) {
            scheduledExecutorService.shutdownNow();
            try {
                schedulerTerminated =
                        scheduledExecutorService.awaitTermination(2, TimeUnit.SECONDS);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        }
        try {
            if (schedulerTerminated) {
                // Flush all remaining events before closing.
                reportFromRingbuffer();
            } else {
                log.warn("Timed out waiting for http report scheduler to stop");
            }
        } catch (HazelcastInstanceNotActiveException ignore) {
            // Hazelcast is shutting down, ringbuffer is not available. Flush from local buffer.
        } catch (IOException e) {
            log.error("Failed to flush events from ringbuffer on close", e);
        }
        try {
            reportFromLocalBuffer();
        } catch (IOException e) {
            log.error("Failed to flush events from local buffer on close", e);
        }
    }

    private OkHttpClient createHttpClient() {
        OkHttpClient client = new OkHttpClient();
        client.setConnectTimeout(30, TimeUnit.SECONDS);
        client.setWriteTimeout(10, TimeUnit.SECONDS);
        return client;

View on GitHub (pinned to cf67b549a7)

Solutions

  1. Ensure the HTTP report endpoint is reachable and responds quickly.
  2. Increase the scheduler shutdown timeout if legitimate slow flushes are expected.
  3. Check network/firewall issues between the SeaTunnel node and the reporting server; unreported events are dropped at shutdown.

Example fix

null
Defensive patterns

Strategy: retry

Try / catch

try { handler.close(); } catch (Exception e) { log.warn("HTTP event handler did not shut down cleanly; events may be lost", e); }

Prevention

When it happens

Trigger: JobEventHttpReportHandler.close() is called while the scheduled executor is still busy (e.g. a blocking HTTP POST to the report endpoint, slow network) and does not reach the schedulerTerminated state before the await timeout expires.

Common situations: Cluster shutdown while the report HTTP endpoint is slow or unreachable; large event backlogs making the flush cycle exceed the shutdown wait time.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors


AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/1fd6afd9f043e618. Report an issue: GitHub.