nautechsystems/nautilus_trader · error

Failed to start AX execution task generation: {e}

Error message

Failed to start AX execution task generation: {e}

What it means

Raised in `connect` on the AX execution client when `pending_tasks.start_generation()` fails after a partial-connect teardown. Task generations group per-connection background workers; failing to open a new generation means the execution client cannot start its background task infrastructure and the connect cannot proceed.

Source

Thrown at crates/adapters/architect_ax/src/execution.rs:522

    fn oms_type(&self) -> OmsType {
        self.core.oms_type
    }

    fn get_account(&self) -> Option<AccountAny> {
        self.core.cache().account_owned(&self.core.account_id)
    }

    async fn connect(&mut self) -> anyhow::Result<()> {
        if self.core.is_connected() && self.pending_tasks.is_open() && self.session_tasks.is_open()
        {
            return Ok(());
        }

        if !self.pending_tasks.is_open() || !self.session_tasks.is_open() {
            self.teardown_partial_connect().await?;
            self.pending_tasks.start_generation().map_err(|e| {
                anyhow::anyhow!("Failed to start AX execution task generation: {e}")
            })?;
            self.session_tasks.start_generation().map_err(|e| {
                anyhow::anyhow!("Failed to start AX execution session generation: {e}")
            })?;
        }
        let http_client = self.http_client.clone();
        let ws_orders = self.ws_orders.clone();
        let setup_guard =
            TaskGroupGuard::new(&[&self.session_tasks, &self.pending_tasks], move || {
                http_client.cancel_all_requests();
                ws_orders.begin_shutdown();
            });

        // Reset so requests work after a previous disconnect
        self.http_client.reset_cancellation_token();

        let credential = Credential::resolve(
            self.config.api_key.clone().map(|value| value.into_inner()),

View on GitHub (pinned to 18893faf8b)

Solutions

1. Read the wrapped inner error for the supervisor's specific lifecycle complaint (e.g., already started vs closed).
2. Avoid concurrent connect() calls on the same client; serialize connection lifecycle calls.
3. Fully tear down (both `teardown_partial_connect` and prior generation completion) before attempting reconnect.
4. If the supervisor is irrecoverably closed, recreate the execution client instance instead of reconnecting.
5. Check for bugs where disconnect marked the supervisor closed without allowing a new generation.

Example fix

// before
// concurrent reconnect from two code paths
client.connect().await?;
// after
// guard with a single-flight connect
if client.connect_lock.compare_exchange(false, true, ...).is_ok() {
    let res = client.connect().await;
    client.connect_lock.store(false, Ordering::SeqCst);
    res?;
}

if !client.is_connected() && connect_in_progress.load(Ordering::SeqCst) {
    anyhow::bail!("connect already in progress; skip duplicate attempt");
}

match client.connect().await {
    Err(e) if e.to_string().contains("task generation") => {
        log::error!("task supervisor in bad state ({e:#}); recreate the client instance");
    }
    other => other?,
}

When it happens

Trigger: Calling `connect()` when either task supervisor was found closed (`!is_open()`), partial teardown ran, and then `start_generation()` on the pending-task supervisor fails — e.g., the supervisor was already started, aborted mid-start, or is in an invalid lifecycle state.

Common situations: Rapid disconnect/reconnect cycles racing the task supervisors' lifecycle; previous shutdown left the supervisor in a closed state that cannot be reopened; concurrent connect attempts on the same client instance.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Cannot connect while previous WebSocket handler task is stil
• Already running
• Cannot set cache database while node is running, set it befo
• Cannot add actor while node is running, add actors before ru
• Cannot add strategy while node is running, add strategies be

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/971e26fefe107eef. Report an issue: GitHub.