nautechsystems/nautilus_trader · error · anyhow::Error

failed to finish Binance Spot data command tasks: {e}

Error message

failed to finish Binance Spot data command tasks: {e}

What it means

When a Binance Spot data client's connect fails partially, teardown_partial_connect aborts the session and command task groups and waits (1s abort / 2s join) for them to finish. If the command task group's finish_shutdown returns an error — tasks that neither completed nor aborted in time, or a join failure — the error is collected and, together with any session-task error, the teardown bails with this joined message.

Source

Thrown at crates/adapters/binance/src/spot/data.rs:306

                .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2)),
            self.command_tasks
                .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2)),
        );
        let mut errors = Vec::new();
        if let Err(e) = session_result {
            errors.push(format!(
                "failed to finish Binance Spot data session tasks: {e}"
            ));
        }

        if let Err(e) = command_result {
            errors.push(format!(
                "failed to finish Binance Spot data command tasks: {e}"
            ));
        }

        if !errors.is_empty() {
            anyhow::bail!(errors.join("; "));
        }
        Ok(())
    }

    async fn prepare_task_groups(&mut self) -> anyhow::Result<()> {
        if !self.session_tasks.is_open() || !self.command_tasks.is_open() {
            self.teardown_partial_connect().await?;
            self.session_tasks
                .start_generation()
                .context("failed to start Binance Spot data session task generation")?;
            self.command_tasks
                .start_generation()
                .context("failed to start Binance Spot data command task generation")?;
            self.cancellation_token = self.session_tasks.cancellation_token();
        }
        Ok(())
    }

View on GitHub (pinned to 18893faf8b)

Solutions

1. Look at the full joined message (it may include the session-task error too) and the log lines from the failing spawned future for the root cause
2. Make spawned command futures cancellation-aware (tokio::select! on the group's cancellation token) so they exit promptly on begin_shutdown
3. Check network path (proxy/DNS) for hangs; add connect/IO timeouts to outbound calls so tasks cannot block indefinitely
4. If it recurs under load, reduce worker starvation (avoid blocking calls on tokio workers) and retry the connect

Example fix

// before: command future blocks on network call without observing shutdown
async move { ws.send(subscribe_msg).await?; Ok(()) }
// after: bound the call and respect the shutdown token
async move {
    tokio::select! {
        _ = cancel.cancelled() => Ok(()),
        r = tokio::time::timeout(Duration::from_secs(5), ws.send(subscribe_msg)) => r.map_err(Into::into),
    }
}

// Verify network reachability and timeouts before connect to reduce hung tasks:
curl -m 5 -sS https://api.binance.com/api/v3/ping && echo "binance spot reachable"

// The teardown error surfaces from connect; catch and inspect the joined message:
match client.connect().await {
    Err(e) if e.to_string().contains("failed to finish Binance Spot data command tasks") => {
        log::warn!("partial-connect teardown timed out; safe to retry connect: {e:#}");
    }
    r => r?,
}

When it happens

Trigger: A connect attempt fails partway (e.g. WebSocket handshake error) triggering teardown_partial_connect, and finish_shutdown on command_tasks fails: spawned command futures ignore the abort signal past the 2s join deadline, or a task panics/returns a join error.

Common situations: WebSocket/REST futures blocked on a hung network call without honoring cancellation; runtime worker starvation under load; a spawned task panicking during shutdown; slow DNS or proxy stalls keeping futures alive past the timeout.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• failed to finish Binance Futures data command tasks: {e}
• old user stream dispatch task did not stop after abort
• Hyperliquid WebSocket handler failed: {error}
• Hyperliquid WebSocket handler did not stop after abort
• Failed to terminate Binance Futures session tasks: {e}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/0fa5a181a949e86c. Report an issue: GitHub.