GET /running-jobs slow diagnostics: full=

Error message

GET /running-jobs slow diagnostics: full={} costMs={} thread={} heapUsedMB={} heapTotalMB={} heapMaxMB={}

What it means

Extended diagnostics warning emitted by RunningJobsServlet.doGet when handling exceeds 500ms: it reports thread name plus heapUsedMB, heapTotalMB, heapMaxMB (from Runtime.totalMemory()-freeMemory()) to pinpoint memory pressure as the slowness cause.

Solutions

Compare heapUsedMB to heapMaxMB; if near saturation, raise -Xmx or reduce cluster load
Capture a GC log / heap dump at the time of the warning
Use paginated /running-jobs requests to shrink handler work
Check for other memory-heavy operations running concurrently on the node

Defensive patterns

Strategy: validation

Validate before calling

// parse the diagnostics pattern to detect heap saturation
const m = logLine.match(/heapUsedMB=(\d+).*heapMaxMB=(\d+)/);
if (m && Number(m[1]) / Number(m[2]) > 0.85) {
  console.error('Coordinator heap near max — increase -Xmx or scale out');
}

Prevention

Keep coordinator heap utilization below ~70-80%
Set -Xmx based on job count and REST load
Enable GC logs to catch full-GC pauses early
Run periodic load tests to find the slow-handler threshold of your cluster

When it happens

Trigger: Same trigger as the 'GET /running-jobs slow' warning — costMs > 500 — with a heap snapshot computed at log time; usually seen alongside GC pressure or large job aggregations.

Common situations: Heap near max causing full GC pauses; very large running-job sets being collected and serialized; node overloaded by other engine tasks.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/c308515db47b67a3. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/servlet/RunningJobsServlet.java:90

        long costMs = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNs);
        long dispatchDelayMs = receivedMs <= 0 ? -1 : Math.max(0, nowMs - receivedMs);

        resp.setHeader("X-Dispatch-Delay-Ms", String.valueOf(dispatchDelayMs));
        resp.setHeader("X-Handler-Cost-Ms", String.valueOf(costMs));

        writeJsonWithPagination(req, resp, runningJobs);

        if (dispatchDelayMs > 500) {
            log.warn(
                    "GET /running-jobs dispatch delayed: dispatchDelayMs={} thread={}",
                    dispatchDelayMs,
                    Thread.currentThread().getName());
        }
        if (costMs > 500) {
            log.warn("GET /running-jobs slow: full={} costMs={}", full, costMs);
            Runtime rt = Runtime.getRuntime();
            long usedBytes = rt.totalMemory() - rt.freeMemory();
            log.warn(
                    "GET /running-jobs slow diagnostics: full={} costMs={} thread={} "
                            + "heapUsedMB={} heapTotalMB={} heapMaxMB={}",
                    full,
                    costMs,
                    Thread.currentThread().getName(),
                    usedBytes / 1024 / 1024,
                    rt.totalMemory() / 1024 / 1024,
                    rt.maxMemory() / 1024 / 1024);
        } else {
            log.debug("GET /running-jobs: full={} costMs={}", full, costMs);
        }
    }
}

View on GitHub (pinned to cf67b549a7)