What causes "Failed to terminate Bybit execution tasks: {e}" in nautechsystems/nautilus_trader?

Retry the disconnect; transient task termination often succeeds on a second attempt

nautechsystems/nautilus_trader · error · anyhow::Error

Failed to terminate Bybit execution tasks: {e}

Error message

Failed to terminate Bybit execution tasks: {e}

What it means

During shutdown, BybitExecutionClient awaits its background (pending) tasks to terminate via finish_shutdown with tight timeouts (1s begin, 2s finish). If tasks do not stop in time or the task group reports an error, the client wraps it in this anyhow error so callers of teardown/connect see shutdown failure explicitly.

Source

Thrown at crates/adapters/bybit/src/execution.rs:269

        if let Err(e) = self.pending_tasks.spawn(future) {
            log::warn!("Skipping Bybit {description} after shutdown began: {e}");
        }
    }

    fn abort_pending_tasks(&self) {
        self.pending_tasks.begin_shutdown();
    }

    fn abort_session_tasks(&self) {
        self.session_tasks.begin_shutdown();
    }

    async fn await_pending_tasks(&self) -> anyhow::Result<()> {
        self.pending_tasks.begin_shutdown();
        self.pending_tasks
            .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2))
            .await
            .map_err(|e| anyhow::anyhow!("Failed to terminate Bybit execution tasks: {e}"))?;
        Ok(())
    }

    async fn await_session_tasks(&self) -> anyhow::Result<()> {
        self.session_tasks.begin_shutdown();
        self.session_tasks
            .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2))
            .await
            .map_err(|e| {
                anyhow::anyhow!("Failed to terminate Bybit execution session tasks: {e}")
            })?;
        Ok(())
    }

    async fn teardown_partial_connect(&mut self) -> anyhow::Result<()> {
        self.abort_session_tasks();
        self.abort_pending_tasks();
        self.http_client.cancel_all_requests();

View on GitHub (pinned to 18893faf8b)

Solutions

Retry the disconnect; transient task termination often succeeds on a second attempt
Check task handlers for blocking calls that prevent cancellation
Inspect the inner error {e} for whether it was a timeout vs an explicit abort
Report/persist the error since some tasks may linger until process exit

Defensive patterns

Strategy: try-catch

Validate before calling

// Before teardown, ensure no pending callbacks are mid-await:
// e.g. wait for in-flight order update handlers to drain before disconnect.

Try / catch

if let Err(e) = client.disconnect().await {
    if e.to_string().contains("Failed to terminate Bybit execution tasks") {
        tracing::warn!("shutdown timed out: {e}; retrying once");
        tokio::time::sleep(Duration::from_millis(500)).await;
        let _ = client.disconnect().await;
    } else {
        return Err(e);
    }
}

Prevention

Avoid blocking calls inside task handlers
Disconnect on the same runtime that spawned the client
Monitor logs for slow handlers during shutdown

When it happens

Trigger: Calling disconnect/teardown_partial_connect while a pending task (e.g. order update stream) is blocked or slow, exceeding the 1–2 second shutdown windows.

Common situations: Network stalls, deadlocked callbacks, or long-running handlers preventing task cancellation; also flaky teardown during reconnect cycles.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/682281ad3c4df7d7. Report an issue: GitHub.