Hmbown/CodeWhale · error
Fleet task does not exist
Error message
Fleet task {} does not exist What it means
During restart_worker's coordinated restart callback, the manager searches the run's task_specs for the spec matching the task's task_id and fails when no spec matches. The ledger knows the task ran, but the run definition no longer carries a spec with that id — an inconsistency between durable task entries and the run's declared workload.
Solutions
- Restore the original run definition so task_specs contains the referenced task id
- Re-enqueue the task under the current run's spec ids instead of restarting the old one
- Reconcile ledger task ids with run.task_specs before calling restart_worker
- Check whether the run was re-created with new spec ids and restart by the new id
Example fix
// before
let spec = run.task_specs.iter().find(|s| s.id == task_id).ok_or(...)?;
// after
if !run.task_specs.iter().any(|s| s.id == task_id) {
anyhow::bail!("task {task_id} was removed from the run; re-enqueue it");
}
let spec = run.task_specs.iter().find(|s| s.id == task_id).unwrap(); Defensive patterns
Strategy: validation
Validate before calling
fn task_spec_exists(state: &FleetLedgerState, run_id: &str, task_id: &str) -> bool {
state.runs.get(run_id)
.map(|r| r.task_specs.iter().any(|s| s.id == task_id))
.unwrap_or(false)
} Prevention
- Do not rewrite a run's task_specs while its tasks exist in the ledger
- When redefining a run, cancel old tasks first
- Keep task ids stable across run definition edits
When it happens
Trigger: Calling restart_worker(worker_id) when the task's entry.task_id is not present in run.task_specs — e.g. the run manifest was rewritten, tasks were removed, or the wrong task id was recorded in the ledger.
Common situations: Editing a fleet run definition while tasks from an older definition are still in the ledger; copying task entries between runs; hand-editing run JSON.
Understand the failure class
Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.
Related errors
- attempt finalization requires a terminal worker event
- attempt finalization status must be terminal
- attempt receipt generation does not match its lease
- attempt receipt identity does not match its terminal event
- conditional progress append does not accept terminal worker…
AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22).
Data as JSON: /api/errors/37c104f4cd9ffea1.
Report an issue: GitHub.
Appendix: source
Thrown at crates/tui/src/fleet/manager.rs:1119
let restarted = self.ledger.restart_task_if_unchanged_with_callback(
&task.entry.run_id,
&task.entry.task_id,
worker_id,
task.status,
task.entry.attempts,
latest_seq,
heartbeat_at,
&now,
None,
task.entry.attempts,
|| {
if let Some(guard) = coordination_guard.as_mut() {
let task_spec = run
.task_specs
.iter()
.find(|spec| spec.id == task.entry.task_id)
.ok_or_else(|| {
anyhow!("Fleet task {} does not exist", task.entry.task_id)
})?;
self.prepare_registered_restart_generation(
guard, &state, task, task_spec, worker_id,
)?;
}
Ok(())
},
)?;
if !restarted {
bail!("worker {worker_id} task changed before it could be restarted");
}
self.ledger
.update_run_status(&task.entry.run_id, FleetRunStatus::Running, ×tamp())?;
Ok(FleetRestartReport {
run_id: task.entry.run_id.clone(),
max_workers,
inspection: self.inspect_worker(worker_id)?,
})View on GitHub (pinned to 73e0f67d83)