{"record":{"id":"0807d72d3dfdc08d","repo":"apache/seatunnel","slug":"the-job-with-id-s-save-point-failed","errorCode":null,"errorMessage":"The job with id '%s' save point failed","messagePattern":"The job with id '(.+?)' save point failed","errorType":"exception","errorClass":"SavePointFailedException","httpStatus":null,"severity":"error","filePath":"seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/CoordinatorService.java","lineNumber":1523,"sourceCode":"        return state instanceof JobStatus ? (JobStatus) state : null;\n    }\n\n    public PassiveCompletableFuture<Void> savePoint(long jobId) {\n        CompletableFuture<Void> voidCompletableFuture = new CompletableFuture<>();\n        if (!runningJobMasterMap.containsKey(jobId)) {\n            SavePointFailedException exception =\n                    new SavePointFailedException(\n                            \"The job with id '\" + jobId + \"' not running, save point failed\");\n            logger.warning(exception);\n            voidCompletableFuture.completeExceptionally(exception);\n        } else {\n            voidCompletableFuture =\n                    new PassiveCompletableFuture<>(\n                            CompletableFuture.supplyAsync(\n                                    () -> {\n                                        JobMaster runningJobMaster = runningJobMasterMap.get(jobId);\n                                        if (!runningJobMaster.savePoint().join()) {\n                                            throw new SavePointFailedException(\n                                                    \"The job with id '\"\n                                                            + jobId\n                                                            + \"' save point failed\");\n                                        }\n                                        try {\n                                            waitForJobComplete(jobId).get();\n                                        } catch (Throwable e) {\n                                            logger.warning(\n                                                    String.format(\n                                                            \"The job with id '%s' waiting state complete failed\",\n                                                            jobId));\n                                        }\n                                        return null;\n                                    },\n                                    executorService));\n        }\n        return new PassiveCompletableFuture<>(voidCompletableFuture);\n    }","sourceCodeStart":1505,"sourceCodeEnd":1541,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/CoordinatorService.java#L1505-L1541","documentation":"This error is thrown on the stop-with-savepoint path when JobMaster.savePoint() completes with false, meaning the savepoint checkpoint did not succeed. The asynchronous supplyAsync block raises SavePointFailedException with this message, failing the caller's future. The job may still be running; only the savepoint attempt failed.","triggerScenarios":"Calling CoordinatorService.savePoint(jobId) when the triggered checkpoint fails or times out, so runningJobMaster.savePoint().join() returns false.","commonSituations":"Heavy backpressure or large state exceeding the checkpoint timeout; unreachable or slow checkpoint storage (HDFS/S3/OSS); savepoint requested while the job master is restarting or missing.","solutions":["Retry the savepoint request; transient checkpoint failures often succeed on a second attempt.","Increase checkpoint timeout / interval and reduce checkpoint pressure (smaller state, more resources).","Verify checkpoint storage connectivity and write permissions from all cluster nodes.","Check JobMaster/checkpoint logs for the underlying checkpoint failure cause before retrying."],"exampleFix":"// before: checkpoint config prone to timeout\ncheckpoint.interval=1000\ncheckpoint.timeout=30000\n// after: larger timeout for big state\ncheckpoint.interval=60000\ncheckpoint.timeout=300000\n// then retry the API call\njobClient.savePoint(jobId).get(5, TimeUnit.MINUTES);","handlingStrategy":"try-catch","validationCode":"// check the job is running before requesting a savepoint\nJobResult r = jobClient.getJobDetail(jobId);\nif (r == null || r.getJobStatus() != JobStatus.RUNNING) {\n    throw new IllegalStateException(\"job not running; savepoint would fail\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    jobClient.savePoint(jobId).get(5, TimeUnit.MINUTES);\n} catch (Exception e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"save point failed\")) {\n        jobClient.savePoint(jobId).get(5, TimeUnit.MINUTES); // one retry\n    } else { throw e; }\n}","preventionTips":["Set a checkpoint timeout proportional to state size.","Monitor checkpoint success rate and alert on failures.","Ensure checkpoint storage is reachable and writable from all nodes.","Retry savepoint once automatically before failing the stop-with-savepoint workflow."],"tags":["zeta-engine","savepoint","checkpoint"],"backgroundTag":"request-timeout","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}