{"record":{"id":"973d48d0ff7f546a","repo":"Hmbown/CodeWhale","slug":"worker-worker-id-task-changed-before-it-could-be","errorCode":null,"errorMessage":"worker {worker_id} task changed before it could be restarted","messagePattern":"worker (.+?) task changed before it could be restarted","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"crates/tui/src/fleet/manager.rs","lineNumber":1090,"sourceCode":"            task.entry.attempts,\n            || {\n                if let Some(guard) = coordination_guard.as_mut() {\n                    let task_spec = run\n                        .task_specs\n                        .iter()\n                        .find(|spec| spec.id == task.entry.task_id)\n                        .ok_or_else(|| {\n                            anyhow!(\"fleet task {} does not exist\", task.entry.task_id)\n                        })?;\n                    self.prepare_registered_restart_generation(\n                        guard, &state, task, task_spec, worker_id,\n                    )?;\n                }\n                Ok(())\n            },\n        )?;\n        if !restarted {\n            bail!(\"worker {worker_id} task changed before it could be restarted\");\n        }\n        self.ledger\n            .update_run_status(&task.entry.run_id, FleetRunStatus::Running, &timestamp())?;\n        Ok(FleetRestartReport {\n            run_id: task.entry.run_id.clone(),\n            max_workers,\n            inspection: self.inspect_worker(worker_id)?,\n        })\n    }\n\n    /// Prepare or consume the exact durable launch generation for one Fleet\n    /// retry. The persisted one-generation-ahead record is the prepare marker:\n    /// it is not launchable while the ledger remains on the old attempt, but a\n    /// retry after a crash may validate and consume it idempotently.\n    fn prepare_registered_restart_generation(\n        &self,\n        coordination: &mut SubAgentManager,\n        state: &FleetLedgerState,","sourceCodeStart":1072,"sourceCodeEnd":1108,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/0c42157ee52f9d55af2b506d71b46249910f77d3/crates/tui/src/fleet/manager.rs#L1072-L1108","documentation":"restart_worker performs a compare-and-swap style update guarded by ledger sequence numbers; the closure returned restarted=false, meaning the task's lease, attempt, or generation changed between the state snapshot and the committed update. Someone else (worker events, another operator action) mutated the task first.","triggerScenarios":"Calling restart_worker concurrently with task completion, lease expiry/reassignment, or another restart/interrupt on the same worker.","commonSituations":"Restart racing the worker's own completion event; two UI panes restarting the same worker; restart issued just as stale reconciliation reaps the lease.","solutions":["Re-inspect the worker to learn its new task/attempt state, then retry restart_worker if still appropriate","Avoid issuing overlapping control commands for the same worker","If this fires constantly, look for a writer loop that keeps bumping the task generation"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"match manager.restart_worker(&worker_id) {\n    Ok(report) => Ok(report),\n    Err(err) if err.to_string().contains(\"task changed before it could be restarted\") => {\n        let inspection = manager.inspect_worker(&worker_id)?; // see what it is now\n        Ok(/* decide based on inspection */)\n    }\n    Err(err) => Err(err),\n}","preventionTips":["Do not stack restart, interrupt, and stop commands on one worker simultaneously","Re-inspect after any control-command race instead of assuming the old task"],"tags":["fleet","race-condition","optimistic-concurrency","rust","codewhale"],"backgroundTag":"optimistic-concurrency-conflict","analyzedSha":"0c42157ee52f9d55af2b506d71b46249910f77d3","analyzedAt":"2026-08-20T21:50:45.477Z","schemaVersion":2},"datasetVersion":"2026-08-22T04:17:13.399Z"}