{"record":{"id":"52d81d04080c731a","repo":"apache/seatunnel","slug":"running-jobs-summary-slow-diagnostics-totalms","errorCode":null,"errorMessage":"running-jobs summary slow diagnostics: totalMs={} jobs={} decodeMs={} heapUsedMB={} heapTotalMB={} heapMaxMB={}","messagePattern":"running-jobs summary slow diagnostics: totalMs=(.+?) jobs=(.+?) decodeMs=(.+?) heapUsedMB=(.+?) heapTotalMB=(.+?) heapMaxMB=(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/service/JobInfoService.java","lineNumber":256,"sourceCode":"                            .add(RestConstant.JOB_NAME, jobName)\n                            .add(RestConstant.JOB_STATUS, jobStatus)\n                            .add(RestConstant.CREATE_TIME, createTime));\n        }\n\n        if (!runningJobBasicInfoCache.isEmpty() && !jobIds.isEmpty()) {\n            runningJobBasicInfoCache.keySet().removeIf(id -> !jobIds.contains(id));\n        }\n\n        long totalMs = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNs);\n        if (totalMs > 500) {\n            log.warn(\n                    \"running-jobs summary slow: total={}ms jobs={} decode={}ms\",\n                    totalMs,\n                    values.size(),\n                    TimeUnit.NANOSECONDS.toMillis(decodeTotalNs));\n            Runtime rt = Runtime.getRuntime();\n            long usedBytes = rt.totalMemory() - rt.freeMemory();\n            log.warn(\n                    \"running-jobs summary slow diagnostics: totalMs={} jobs={} decodeMs={} \"\n                            + \"heapUsedMB={} heapTotalMB={} heapMaxMB={}\",\n                    totalMs,\n                    values.size(),\n                    TimeUnit.NANOSECONDS.toMillis(decodeTotalNs),\n                    usedBytes / 1024 / 1024,\n                    rt.totalMemory() / 1024 / 1024,\n                    rt.maxMemory() / 1024 / 1024);\n        } else {\n            log.debug(\n                    \"running-jobs summary: total={}ms jobs={} decode={}ms\",\n                    totalMs,\n                    values.size(),\n                    TimeUnit.NANOSECONDS.toMillis(decodeTotalNs));\n        }\n        return out;\n    }\n","sourceCodeStart":238,"sourceCodeEnd":274,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/service/JobInfoService.java#L238-L274","documentation":"Follow-up diagnostics emitted right after the slow-summary warning (errorIndex 3713) when the running-jobs summary exceeds 500ms. It adds heap diagnostics: heap used/total/max in MB, to correlate slowness with memory pressure (GC thrash, near-capacity heap).","triggerScenarios":"Same trigger as the slow summary (total > 500ms); logged whenever heapUsedMB is near heapMaxMB or decode time dominates, indicating GC pressure or payload-heavy decoding.","commonSituations":"REST node running close to -Xmx with heavy dashboard polling; frequent full GCs slowing map iteration and JSON decode; jobs with very large basic-info payloads.","solutions":["Increase the REST/master node -Xmx based on heapUsedMB vs heapMaxMB in the log.","Tune GC (e.g., G1 with shorter pause goals) to reduce decode-time inflation.","Reduce per-job payload size / number of jobs returned per summary call.","Correlate with GC logs; if heapUsedMB oscillates at max, the slowness is GC-driven, not logic-driven."],"exampleFix":"// before\n// JAVA_OPTS=\"-Xms2g -Xmx2g\"\n// after (heap headroom for REST summary endpoint)\n// JAVA_OPTS=\"-Xms4g -Xmx4g -XX:+UseG1GC -XX:MaxGCPauseMillis=200\"","handlingStrategy":"retry","validationCode":"// before bulk REST polling, check headroom\n// Runtime rt = Runtime.getRuntime();\n// double usedRatio = (double)(rt.totalMemory()-rt.freeMemory())/rt.maxMemory();\n// if (usedRatio > 0.85) { reducePolling(); }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Size -Xmx so heap used stays well below max","Enable G1GC with pause targets for REST-facing nodes","Reduce payload size per summary request"],"tags":["performance","jvm","heap","gc","rest-api"],"backgroundTag":"request-timeout","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}