Fleet worker {worker_id} persisted launch generation {genera

Error message

Fleet worker {worker_id} persisted launch generation {generation} does not match ledger attempt {current_generation} or its prepared retry {next_generation}

What it means

Fleet persists a one-generation-ahead 'prepared retry' record whose generation must equal either the ledger's current attempt or the prepared next attempt. The durable record's generation matched neither, so the persisted launch state and the event ledger have diverged. This is a consistency tripwire, not an expected runtime state.

Source

Thrown at crates/tui/src/fleet/manager.rs:1177

                validate_registered_launch_spec(&record.spec, &expected_current)?;
                coordination
                    .advance_registered_worker_generation(bind_fleet_launch_attempt(
                        record.spec,
                        next_generation,
                    ))
                    .map_err(anyhow::Error::msg)?;
            }
            generation if generation == next_generation => {
                let mut normalized = record.spec;
                normalized
                    .launch_manifest
                    .as_mut()
                    .expect("prepared launch manifest checked above")
                    .generation = current_generation;
                validate_registered_launch_spec(&normalized, &expected_current)?;
            }
            generation => {
                bail!(
                    "Fleet worker {worker_id} persisted launch generation {generation} does not match ledger attempt {current_generation} or its prepared retry {next_generation}"
                );
            }
        }
        Ok(())
    }

    pub fn stop_all(&self) -> Result<usize> {
        let state = self.ledger.rebuild_state()?;
        let now = timestamp();
        let mut affected_runs = BTreeSet::new();
        let mut stopped = 0usize;
        for task in state.tasks.values() {
            if !matches!(
                task.status,
                FleetTaskLedgerStatus::Enqueued | FleetTaskLedgerStatus::Leased
            ) {
                continue;

View on GitHub (pinned to 0c42157ee5)

Solutions

Let stale reconciliation run to completion, then retry the restart to see if generations re-align
Clear the stale prepared-retry record for that worker so a fresh generation can be prepared
If the run's durable state is inconsistent, finish/cancel the run and start a new one rather than forcing the old one
Report it as a bug if reproducible without a crash or manual state editing

Defensive patterns

Strategy: fallback

Try / catch

match manager.restart_worker(&worker_id) {
    Ok(report) => Ok(report),
    Err(err) if err.to_string().contains("does not match ledger attempt") => {
        // durable state diverged: retire the run and start fresh rather than forcing it
        let _ = manager.stop_run(&run_id);
        Err(err)
    }
    Err(err) => Err(err),
}

Prevention

Never hand-edit ledger or coordination state files
Restore ledger and coordination state only as a matched pair
Avoid hard-killing Codewhale mid-restart; use stop paths

When it happens

Trigger: A crash between preparing and consuming a retry generation; ledger events partially applied; the coordination store and the ledger updated out of order; state files copied/merged from another run.

Common situations: Hard-killing Codewhale during a worker restart; restoring a run directory from backup where coordination records and ledger events are from different points in time; hand-editing state files.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@0c42157ee5 (2026-08-20). Data as JSON: /api/errors/66434df3603c13ef. Report an issue: GitHub.