apache/seatunnel · error · JobNotFoundException

Job not running

Error message

Job %s not running

What it means

CoordinatorService.updateTaskExecutionState throws JobNotFoundException when neither a restore-in-progress marker nor a runningJobMaster exists for the job of the reported task group. The coordinator has no live JobMaster for this job, so the task state update cannot be applied.

Solutions

1. Treat as benign if the job was cancelled/failed: the task's state update is no longer relevant once the JobMaster is gone.
2. Check coordinator logs for why the JobMaster was released (job finished, failover, manual stop-job) around the same timestamp.
3. If it recurs for live jobs, inspect cluster stability (network partitions, master failover storms) and ensure all nodes run the same SeaTunnel version.

// before sending a state update, confirm the job is still live on the coordinator
// (poll running-jobs / check job state via the client API first)

try {
    coordinator.updateTaskExecutionState(state);
} catch (JobNotFoundException e) {
    log.info("Job {} already released; dropping late task state update", state.getJobId());
}

When it happens

Trigger: A worker sends UpdateTaskExecutionStateOperation for a task group whose jobId has no JobMaster on the coordinator — after the job was failed/cancelled and its JobMaster released, after job cleanup during failover, or if the coordinator restarted without restoring that job.

Common situations: Late task-state reports racing with job cancellation/termination; stale workers continuing to report after their job was removed during master failover; clock/binding mismatch where the worker was not told the job ended.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

• Job not found
• task group not found.
• Can not get coordinator service from an active master node.
• cluster have no master node
• Job not running (restore in progress)

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/f9c9fe004f58d1d0. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/CoordinatorService.java:2080

                        taskExecutionState.getTaskGroupLocation(),
                        taskExecutionState.getExecutionState()));
        TaskGroupLocation taskGroupLocation = taskExecutionState.getTaskGroupLocation();
        JobMaster runningJobMaster = runningJobMasterMap.get(taskGroupLocation.getJobId());

        if (runningJobMaster == null && !restoreAllJobFromMasterNodeSwitchFuture.isDone()) {
            // Restore still in progress, return early and let worker retry
            // This is acceptable because worker already has retry logic
            logger.info(
                    String.format(
                            "Job %s not found and restore still in progress, worker will retry",
                            taskGroupLocation.getJobId()));
            throw new JobRestoreInProgressException(
                    String.format(
                            "Job %s not running (restore in progress)",
                            taskGroupLocation.getJobId()));
        }
        if (runningJobMaster == null) {
            throw new JobNotFoundException(
                    String.format("Job %s not running", taskGroupLocation.getJobId()));
        }
        runningJobMaster.updateTaskExecutionState(taskExecutionState);
    }

    public void shutdown() {
        isActive = false;
        if (masterActiveListener != null) {
            masterActiveListener.shutdown();
        }
        if (metricsFetchExecutor != null) {
            metricsFetchExecutor.shutdownNow();
        }
        if (pipelineCleanupScheduler != null) {
            pipelineCleanupScheduler.shutdown();
        }
        clearCoordinatorService();
        awaitSchedulerTermination("master active listener", masterActiveListener);

View on GitHub (pinned to cf67b549a7)