nautechsystems/nautilus_trader · error

Failed to terminate AX execution session tasks

Error message

Failed to terminate AX execution session tasks: {e}

What it means

Raised in `await_session_tasks` when the execution client's session-task supervisor (`session_tasks`) fails to finish shutdown within its 1s grace and 2s hard deadline. Session tasks (e.g., session keep-alive/streams for the AX execution client) did not terminate cleanly during teardown.

Solutions

1. Inspect the wrapped inner error for which session tasks did not exit.
2. Ensure session task loops select on the shutdown/cancellation signal and exit promptly.
3. Add timeouts to session network reads so tasks cannot block indefinitely.
4. Increase the grace/timeout durations if sessions need more time to wind down.
5. Verify a fallback abort path (e.g., `abort_session_tasks`) is invoked after the failure to avoid leaked tasks.

Example fix

// before
loop {
    let msg = ws.next().await;  // blocks shutdown
    handle(msg);
}
// after
loop {
    tokio::select! {
        msg = ws.next() => handle(msg),
        _ = shutdown.cancelled() => break,
    }
}

match client.await_session_tasks().await {
    Err(e) => {
        log::warn!("session tasks shutdown timed out: {e:#}");
        client.abort_session_tasks(); // forced fallback
    }
    Ok(()) => {},
}

When it happens

Trigger: Calling `disconnect()`/`await_session_tasks` when session tasks remain blocked — typically a WebSocket session loop or stream reader stuck on a hung connection that ignores the shutdown signal until the hard deadline passes.

Common situations: WS session loop not observing the cancellation token; TCP connection half-open during teardown; slow AX server responses keeping session handlers alive; shutdown racing with an active reconnect loop.

Related errors

• Failed to terminate AX data session tasks
• Failed to terminate AX data tasks
• Failed to terminate AX execution tasks
• Failed to terminate Betfair session tasks
• Architect AX data WebSocket handler did not stop after abort

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/c7474e038bb1aa96. Report an issue: GitHub.

Appendix: source

Thrown at crates/adapters/architect_ax/src/execution.rs:425

        self.session_tasks.begin_shutdown();
        self.ws_orders.begin_shutdown();
    }

    async fn await_pending_tasks(&self) -> anyhow::Result<()> {
        self.pending_tasks.begin_shutdown();
        self.pending_tasks
            .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2))
            .await
            .map_err(|e| anyhow::anyhow!("Failed to terminate AX execution tasks: {e}"))?;
        Ok(())
    }

    async fn await_session_tasks(&self) -> anyhow::Result<()> {
        self.session_tasks.begin_shutdown();
        self.session_tasks
            .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2))
            .await
            .map_err(|e| anyhow::anyhow!("Failed to terminate AX execution session tasks: {e}"))?;
        Ok(())
    }

    async fn teardown_partial_connect(&mut self) -> anyhow::Result<()> {
        self.abort_session_tasks();
        self.abort_pending_tasks();
        self.http_client.cancel_all_requests();

        if let Err(e) = self.ws_orders.close().await {
            self.shutdown_errors
                .push(format!("AX orders WebSocket shutdown failed: {e}"));
        }

        let (session_result, pending_result) =
            tokio::join!(self.await_session_tasks(), self.await_pending_tasks());
        self.core.set_disconnected();

        if let Err(e) = session_result {

View on GitHub (pinned to 18893faf8b)