fleet executor outcome must contain a terminal worker event

Error message

fleet executor outcome must contain a terminal worker event

What it means

The terminal-outcome handler destructures a FleetExecutorTerminalEvent and requires payload to be Completed, Failed, or Cancelled; anything else hits the 'must contain a terminal worker event' bail. This is an internal invariant violation: the executor emitted an outcome labelled terminal whose payload is not a terminal event type.

Source

Thrown at crates/tui/src/fleet/manager.rs:1654

            || current.entry.attempts != task.entry.attempts
        {
            return Ok(false);
        }

        let FleetWorkerTerminalEvent {
            payload,
            exit_code,
            tail_payloads,
            reported_route,
            requires_reported_route,
        } = terminal;
        let (receipt_result, failure_kind, exit_code) = task_receipt_outcome(&payload, exit_code);
        let terminal_completed = matches!(&payload, FleetWorkerEventPayload::Completed { .. });
        let expected_terminal_status = match &payload {
            FleetWorkerEventPayload::Completed { .. } => FleetTaskLedgerStatus::Completed,
            FleetWorkerEventPayload::Failed { .. } => FleetTaskLedgerStatus::Failed,
            FleetWorkerEventPayload::Cancelled { .. } => FleetTaskLedgerStatus::Cancelled,
            _ => bail!("fleet executor outcome must contain a terminal worker event"),
        };
        for tail_payload in tail_payloads {
            if is_terminal_payload(&tail_payload) {
                continue;
            }
            if self
                .ledger
                .append_event_if_leased(
                    &task.entry.run_id,
                    &task.worker_id,
                    &task.entry.task_id,
                    task.entry.attempts,
                    &timestamp(),
                    tail_payload,
                )?
                .is_none()
            {
                return Ok(false);

View on GitHub (pinned to 0c42157ee5)

Solutions

Ensure the manager and all workers (including SSH remotes) run the same Codewhale version
Restart the affected run with a single consistent binary
Collect the ledger events around the failure and report it as a bug with versions attached

Defensive patterns

Strategy: try-catch

Type guard

fn payload_is_terminal(payload: &FleetWorkerEventPayload) -> bool {
    matches!(
        payload,
        FleetWorkerEventPayload::Completed { .. }
            | FleetWorkerEventPayload::Failed { .. }
            | FleetWorkerEventPayload::Cancelled { .. }
    )
}

Try / catch

match handle_terminal(outcome) {
    Ok(receipt) => Ok(receipt),
    Err(err) if err.to_string().contains("must contain a terminal worker event") => {
        // log ledger context + versions and report upstream; do not retry blindly
        Err(err)
    }
    Err(err) => Err(err),
}

Prevention

Pin matching Codewhale versions on the manager and all SSH remote workers
Include binary versions in fleet logs to diagnose skew quickly
Never construct terminal outcomes with non-terminal payloads in custom executors

When it happens

Trigger: Produced only by a mismatch between the executor and manager: a version-skewed worker/executor sending a non-terminal payload in the terminal slot, or a bug in outcome construction. No user configuration causes it.

Common situations: Running mixed Codewhale binary versions in a fleet (SSH remote workers older than the local manager); a custom build patched to add a new event type without updating this match.

Related errors

AI-assisted analysis of Hmbown/CodeWhale@0c42157ee5 (2026-08-20). Data as JSON: /api/errors/3037390cc1ae6388. Report an issue: GitHub.