Running job dag json cache evicted entries due to max size:

Error message

Running job dag json cache evicted entries due to max size: evicted={}, expiredRemoved={}, remaining={}, background={}

What it means

BaseService maintains an in-memory cache of running-job DAG JSON for the REST API. cleanupRunningJobDagCache() evicts entries when the cache exceeds max size or expires. When entries were evicted due to max size, a rate-limited warning (via RUNNING_JOB_DAG_CACHE_LAST_LOG_MS CAS) reports eviction statistics. It signals cache pressure — the cache max size was hit and oldest/expired entries were dropped; they are recomputable on demand.

Source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/service/BaseService.java:305

                    needEvict--;
                    evicted++;
                }
            }
        }

        if (expiredRemoved <= 0 && evicted <= 0) {
            return;
        }
        long lastLog = RUNNING_JOB_DAG_CACHE_LAST_LOG_MS.get();
        if (nowMs - lastLog < RUNNING_JOB_DAG_CACHE_CLEANUP_INTERVAL_MS) {
            return;
        }
        if (!RUNNING_JOB_DAG_CACHE_LAST_LOG_MS.compareAndSet(lastLog, nowMs)) {
            return;
        }

        if (evicted > 0) {
            log.warn(
                    "Running job dag json cache evicted entries due to max size: evicted={}, expiredRemoved={}, remaining={}, background={}",
                    evicted,
                    expiredRemoved,
                    RUNNING_JOB_DAG_JSON_CACHE.size(),
                    background);
        } else {
            log.debug(
                    "Running job dag json cache cleanup: expiredRemoved={}, remaining={}, background={}",
                    expiredRemoved,
                    RUNNING_JOB_DAG_JSON_CACHE.size(),
                    background);
        }
    }

    protected SeaTunnelServer getSeaTunnelServer(boolean shouldBeMaster) {
        SeaTunnelServer seaTunnelServer = null;
        try {
            com.hazelcast.instance.impl.NodeExtension nodeExtension =

View on GitHub (pinned to cf67b549a7)

Solutions

If job count is legitimately high, increase the cache max size configuration for the running-job DAG cache.
Treat as informational if evicted entries are stale/expired — correctness is unaffected (cache is recomputed).
Reduce REST polling frequency of DAG endpoints if load-driven.
Monitor remaining= count; if it stays at max, tune the cache sizing parameters.

Defensive patterns

Strategy: fallback

Validate before calling

// size the cache based on concurrent running-job count
int runningJobs = jobMasterList.size();
int recommendedMax = Math.max(configuredMax, runningJobs);

Prevention

Set the running-job DAG cache max size at or above expected concurrent job count
Reduce REST dashboard polling frequency
Monitor eviction logs as a capacity signal
Rely on the cache being recomputable — treat evictions as non-fatal

When it happens

Trigger: Number of running jobs whose DAG JSON is cached exceeds the configured RUNNING_JOB_DAG_JSON_CACHE max size; cleanup runs on access or from a background cleaner and evicts entries.

Common situations: Large clusters running many concurrent jobs; long-lived jobs keeping cache entries alive; REST metrics/dashboard endpoints polled frequently generating many cache entries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/f05187d0ad5c0cc0. Report an issue: GitHub.