apache/seatunnel · critical · RuntimeException

Failed to load readyToCloseStartingTask from IMap, key: %s

Error message

Failed to load readyToCloseStartingTask from IMap, key: %s

What it means

CheckpointCoordinator.loadReadyToCloseStartingTask reads the readyToCloseStartingTask marker from the distributed IMap; if any exception occurs while loading, it logs the job/pipeline IDs and rethrows a RuntimeException wrapping the cause. This marker records whether the starting (source) task is ready to close, and failing to load it means the coordinator cannot safely continue the checkpoint/close protocol.

Source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/checkpoint/CheckpointCoordinator.java:579

        try {
            Object stored = runningJobStateIMap.get(readyToCloseImapKey);
            if (stored instanceof Set) {
                Set<TaskLocation> result = (Set<TaskLocation>) stored;
                LOG.info(
                        "Loaded readyToCloseStartingTask from IMap, job id: {}, pipeline id: {}, value: {}",
                        jobId,
                        pipelineId,
                        result);
                return result;
            }
            return null;
        } catch (Exception e) {
            LOG.error(
                    "Failed to load readyToCloseStartingTask from IMap, job id: {}, pipeline id: {}.",
                    jobId,
                    pipelineId,
                    e);
            throw new RuntimeException(
                    "Failed to load readyToCloseStartingTask from IMap, key: "
                            + readyToCloseImapKey,
                    e);
        }
    }

    private void updateReadyToCloseStartingTask() {
        try {
            RetryUtils.retryWithException(
                    () -> {
                        runningJobStateIMap.compute(
                                readyToCloseImapKey,
                                (k, exist) -> {
                                    Set<TaskLocation> merged =
                                            exist instanceof Set
                                                    ? new HashSet<>((Set<TaskLocation>) exist)
                                                    : new HashSet<>();
                                    merged.addAll(readyToCloseStartingTask);

View on GitHub (pinned to cf67b549a7)

Solutions

1. Inspect the wrapped cause (getCause()) — fix the underlying Hazelcast/IMap issue (timeouts, connectivity) it reports
2. Retry the job restore; transient Hazelcast timeouts and migration pauses usually resolve on re-run
3. Check cluster stability and Hazelcast logs for partition migration or split-brain around the failure time
4. If it follows a version upgrade, verify checkpoint/state storage format compatibility between old and new versions
5. Ensure IMap persistence/backup config is consistent so the key exists after failover

Example fix

// before
try {
    loadReadyToCloseStartingTask();
} catch (Exception e) {
    // swallowed or generic handling
}

// after
try {
    loadReadyToCloseStartingTask();
} catch (RuntimeException e) {
    LOG.error("readyToClose load failed, key={}", readyToCloseImapKey, e.getCause());
    throw e; // fail job; retry on resubmit after fixing IMap/cluster issue
}

// pre-check key presence
boolean present = hazelcastInstance.getMap(mapName).containsKey(readyToCloseImapKey);

try {
    loadReadyToCloseStartingTask();
} catch (RuntimeException e) {
    if (e.getCause() instanceof HazelcastInstanceNotActiveException || e.getCause() instanceof OperationTimeoutException) {
        retryWithBackoff();
    } else { throw e; }
}

When it happens

Trigger: IMap read failure during restoredReadyToClose: Hazelcast operation timeout, partition migration/failover while reading, deserialization failure of the stored value, or cluster connection loss during the get() call.

Common situations: Network instability between master node and Hazelcast cluster during job restore; master failover mid-checkpoint; corrupted or incompatible serialized state after a SeaTunnel version upgrade.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

• Failed to persist readyToCloseStartingTask to IMap, key: %s
• Failed to fetch running jobs from IMap during master switch
• Job id %s restore failed, can not get job state
• The checkpoint coordinator(%s) don't exist
• Unknown checkpoint type:

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/9f6d5a52ea4e953d. Report an issue: GitHub.