{"record":{"id":"44f30f10b3036f1f","repo":"zeroclaw-labs/zeroclaw","slug":"run-cannot-resume-its-parked-checkpoint-snapsh","errorCode":null,"errorMessage":"Run {} cannot resume: its parked checkpoint snapshot is not yet durably persisted (retrying)","messagePattern":"Run (.+?) cannot resume: its parked checkpoint snapshot is not yet durably persisted \\(retrying\\)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/zeroclaw-runtime/src/sop/engine.rs","lineNumber":4333,"sourceCode":"                let run_id = state.run_id.clone();\n                ::zeroclaw_log::record!(\n                    WARN,\n                    ::zeroclaw_log::Event::new(module_path!(), ::zeroclaw_log::Action::Reject)\n                        .with_outcome(::zeroclaw_log::EventOutcome::Failure)\n                        .with_attrs(::serde_json::json!({\"run_id\": run_id})),\n                    \"SOP engine: active run not found\"\n                );\n                bail!(\"Active run not found: {}\", state.run_id);\n            }\n        };\n\n        // Refuse to resume while the checkpoint's parked snapshot has not yet\n        // been durably persisted (see `is_park_persist_pending`'s doc): the kept\n        // claim predates this attempt, and reacquiring on top of it would give a\n        // later rollback or a maintenance retry no way to distinguish \"freshly\n        // reacquired\" from \"pre-existing, must survive.\"\n        if self.is_park_persist_pending(&state.run_id) {\n            bail!(\n                \"Run {} cannot resume: its parked checkpoint snapshot is not yet durably persisted (retrying)\",\n                state.run_id\n            );\n        }\n\n        let sop = self\n            .sops\n            .iter()\n            .find(|s| s.name == sop_name)\n            .ok_or_else(|| {\n                ::zeroclaw_log::record!(\n                    WARN,\n                    ::zeroclaw_log::Event::new(module_path!(), ::zeroclaw_log::Action::Fail)\n                        .with_outcome(::zeroclaw_log::EventOutcome::Failure)\n                        .with_attrs(::serde_json::json!({\"sop_name\": sop_name.as_str()})),\n                    \"SOP engine: sop no longer loaded (definition removed mid-run)\"\n                );\n                anyhow::Error::msg(format!(\"SOP '{sop_name}' no longer loaded\"))","sourceCodeStart":4315,"sourceCodeEnd":4351,"githubUrl":"https://github.com/zeroclaw-labs/zeroclaw/blob/88bb9c8533fc57ed7a03e36ca7c9ed2bf8336dcc/crates/zeroclaw-runtime/src/sop/engine.rs#L4315-L4351","documentation":"Resume refuses to proceed while is_park_persist_pending(run_id) is true: the run's exec claim predates this attempt, and reacquiring on top of an unpersisted park would let a later rollback or maintenance retry confuse 'freshly reacquired' with 'pre-existing, must survive'. The guard fails closed; a maintenance tick's retry durably persists the park, after which resume works.","triggerScenarios":"Resuming immediately after a run parks at a checkpoint while its park snapshot write is still outstanding (slow disk, store latency, or a failed persist being retried).","commonSituations":"Automated resume pipelines reacting instantly to a PausedCheckpoint status; degraded durable storage slowing the park write; heavy fsync load delaying the snapshot.","solutions":["Retry resume with backoff; the condition is transient and clears once the park is durably persisted.","If retries keep failing, inspect durable-store health and engine logs for park-persist retry failures.","Do not strip or work around the guard; it protects claim accounting during the pending window."],"exampleFix":"// before: immediate resume after seeing PausedCheckpoint\nengine.resume_checkpoint(state).await?; // trips while park persist is in flight\n\n// after: bounded retry until the snapshot is durable\nlet mut attempt = 0u32;\nloop {\n    match engine.resume_checkpoint(state.clone()).await {\n        Ok(out) => break out,\n        Err(e) if e.to_string().contains(\"not yet durably persisted\") && attempt < 8 => {\n            attempt += 1;\n            tokio::time::sleep(std::time::Duration::from_millis(250 * u64::from(attempt))).await;\n        }\n        Err(e) => return Err(e),\n    }\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"let mut attempt = 0u32;\nloop {\n    match engine.resume_checkpoint(state.clone()).await {\n        Ok(out) => break out,\n        Err(e) if e.to_string().contains(\"not yet durably persisted\") && attempt < 8 => {\n            attempt += 1;\n            tokio::time::sleep(std::time::Duration::from_millis(250 * u64::from(attempt))).await;\n        }\n        Err(e) => return Err(e),\n    }\n}","preventionTips":["Delay resume slightly after observing PausedCheckpoint so the park persist can land.","Watch for park-persist retry failures in logs when resumes keep bouncing.","Never acquire claims manually to work around the pending window."],"tags":["sop","checkpoint","resume","persistence","transient"],"backgroundTag":"eventual-consistency-retry","analyzedSha":"88bb9c8533fc57ed7a03e36ca7c9ed2bf8336dcc","analyzedAt":"2026-08-23T01:07:41.857Z","schemaVersion":2},"datasetVersion":"2026-08-23T08:06:27.607Z"}