nautechsystems/nautilus_trader · warning

Failed to terminate AX data session tasks: {e}

Error message

Failed to terminate AX data session tasks: {e}

What it means

finish_all_tasks also awaits session_tasks' graceful shutdown with 1s/2s timeouts. If a session task (e.g. session generation/keepalive task) fails to terminate in time, the error is wrapped as "Failed to terminate AX data session tasks". It indicates leftover session-level tasks after disconnect.

Source

Thrown at crates/adapters/architect_ax/src/data.rs:366

        for cancellation in self.funding_rate_cancellations.values() {
            cancellation.cancel();
        }
    }

    async fn finish_all_tasks(&mut self) -> anyhow::Result<()> {
        self.pending_tasks.begin_shutdown();
        self.session_tasks.begin_shutdown();
        let (pending_result, session_result) = tokio::join!(
            self.pending_tasks
                .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2)),
            self.session_tasks
                .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2)),
        );
        self.funding_rate_cancellations.clear();

        pending_result.map_err(|e| anyhow::anyhow!("Failed to terminate AX data tasks: {e}"))?;
        session_result
            .map_err(|e| anyhow::anyhow!("Failed to terminate AX data session tasks: {e}"))?;
        Ok(())
    }

    async fn teardown_partial_connect(&mut self) -> anyhow::Result<()> {
        self.abort_all_tasks();

        if let Err(e) = self.ws_client.close().await {
            self.shutdown_errors.push(e.to_string());
        }

        if let Err(e) = self.finish_all_tasks().await {
            self.shutdown_errors.push(e.to_string());
        }
        self.is_connected.store(false, Ordering::Release);

        if !self.shutdown_errors.is_empty() {
            anyhow::bail!(std::mem::take(&mut self.shutdown_errors).join("; "));
        }

View on GitHub (pinned to 18893faf8b)

Solutions

1. Force-abort session tasks if graceful shutdown repeatedly times out
2. Verify session tasks respect the cancellation token and wake promptly
3. Check network state; retry disconnect once connectivity is restored
4. Increase shutdown timeout durations if graceful wind-down legitimately needs more time

Example fix

// before
self.session_tasks.finish_shutdown(Duration::from_secs(1), Duration::from_secs(2))
// after
self.session_tasks.abort_all(); // then finish_shutdown for bookkeeping
self.session_tasks.finish_shutdown(Duration::from_secs(1), Duration::from_secs(2))

// Rust
// ensure cancellation token is cancelled so session tasks can exit before shutdown await

// Rust
match client.disconnect().await {
    Err(e) if e.to_string().contains("session tasks") => log::warn!("session tasks hung; ignoring during shutdown"),
    other => other?,
}

When it happens

Trigger: Calling disconnect or teardown_partial_connect while AX session tasks remain alive past the finish_shutdown timeouts — e.g. a session task blocked on a hung websocket read.

Common situations: Disconnected network during shutdown; session task waiting on a server response that never arrives; task not observing the cancellation token.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• Failed to terminate AX data tasks: {e}
• Hyperliquid WebSocket handler did not stop after abort
• Failed to terminate AX execution tasks: {e}
• Failed to terminate AX execution session tasks: {e}
• Architect AX data WebSocket handler did not stop after abort

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/b363c819c4503308. Report an issue: GitHub.