apache/seatunnel · error · IllegalStateException

Failed to get checkpoint data from master node, restoreSourc

Error message

Failed to get checkpoint data from master node, restoreSourceJobId=${restoreSourceJobId}

What it means

On non-master nodes, checkpoint data is fetched from the master via sendOperationToMasterNode. Any exception during that RPC or during deserialization of the response into List<JobPipelineCheckpointData> is wrapped in an IllegalStateException mentioning the restoreSourceJobId.

Source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/RestJobExecutionEnvironment.java:197

                        .getLatestCheckpointData(String.valueOf(restoreSourceJobId), restoreMode);
            }
            throw new IllegalStateException(
                    "Unsupported restore mode for checkpoint loading: " + restoreMode);
        }

        try {
            Object response =
                    NodeEngineUtil.sendOperationToMasterNode(
                                    nodeEngine,
                                    new GetJobCheckpointOperation(restoreSourceJobId, restoreMode))
                            .join();
            if (response == null) {
                return Collections.emptyList();
            }
            return (List<JobPipelineCheckpointData>)
                    nodeEngine.getSerializationService().toObject(response);
        } catch (Exception e) {
            throw new IllegalStateException(
                    "Failed to get checkpoint data from master node, restoreSourceJobId="
                            + restoreSourceJobId,
                    e);
        }
    }

    public JobImmutableInformation build() {
        return new JobImmutableInformation(
                Long.parseLong(jobConfig.getJobContext().getJobId()),
                jobConfig.getName(),
                restoreMode,
                restoreSourceJobId,
                nodeEngine.getSerializationService(),
                getLogicalDag(),
                new ArrayList<>(jarUrls),
                new ArrayList<>(connectorJarIdentifiers));
    }
}

View on GitHub (pinned to cf67b549a7)

Solutions

Check master node availability and network connectivity between nodes; retry the submission
Verify all nodes run the same SeaTunnel version (serialization compatibility)
Ensure checkpoint storage is accessible from the master node and credentials are valid
Inspect master node logs at the same timestamp for the root cause exception

Defensive patterns

Strategy: retry

Validate before calling

// confirm master reachable before submission
curl http://master:8080/hazelcast/rest/maps/cluster-overview

Try / catch

try {
    loadRemoteCheckpoints();
} catch (IllegalStateException e) {
    if (e.getMessage().startsWith("Failed to get checkpoint data")) {
        // check master logs / connectivity, then retry
    }
}

Prevention

Monitor master node health and network partition risks
Keep all cluster nodes on the same version
Ensure checkpoint storage HA (HDFS/S3) rather than local disk

When it happens

Trigger: Master node unreachable/slow during restore submission; operation timeout; serialization-service mismatch between nodes; master throws while reading checkpoint from storage.

Common situations: Cluster network issues or master restarted mid-request; version-skew between cluster nodes; checkpoint storage (HDFS/S3) down so master cannot read state.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/087b804b1498a607. Report an issue: GitHub.