{"record":{"id":"0fa5a181a949e86c","repo":"nautechsystems/nautilus_trader","slug":"failed-to-finish-binance-spot-data-command-tasks","errorCode":null,"errorMessage":"failed to finish Binance Spot data command tasks: {e}","messagePattern":"failed to finish Binance Spot data command tasks: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/adapters/binance/src/spot/data.rs","lineNumber":306,"sourceCode":"                .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2)),\n            self.command_tasks\n                .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2)),\n        );\n        let mut errors = Vec::new();\n        if let Err(e) = session_result {\n            errors.push(format!(\n                \"failed to finish Binance Spot data session tasks: {e}\"\n            ));\n        }\n\n        if let Err(e) = command_result {\n            errors.push(format!(\n                \"failed to finish Binance Spot data command tasks: {e}\"\n            ));\n        }\n\n        if !errors.is_empty() {\n            anyhow::bail!(errors.join(\"; \"));\n        }\n        Ok(())\n    }\n\n    async fn prepare_task_groups(&mut self) -> anyhow::Result<()> {\n        if !self.session_tasks.is_open() || !self.command_tasks.is_open() {\n            self.teardown_partial_connect().await?;\n            self.session_tasks\n                .start_generation()\n                .context(\"failed to start Binance Spot data session task generation\")?;\n            self.command_tasks\n                .start_generation()\n                .context(\"failed to start Binance Spot data command task generation\")?;\n            self.cancellation_token = self.session_tasks.cancellation_token();\n        }\n        Ok(())\n    }\n","sourceCodeStart":288,"sourceCodeEnd":324,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/binance/src/spot/data.rs#L288-L324","documentation":"When a Binance Spot data client's connect fails partially, teardown_partial_connect aborts the session and command task groups and waits (1s abort / 2s join) for them to finish. If the command task group's finish_shutdown returns an error — tasks that neither completed nor aborted in time, or a join failure — the error is collected and, together with any session-task error, the teardown bails with this joined message.","triggerScenarios":"A connect attempt fails partway (e.g. WebSocket handshake error) triggering teardown_partial_connect, and finish_shutdown on command_tasks fails: spawned command futures ignore the abort signal past the 2s join deadline, or a task panics/returns a join error.","commonSituations":"WebSocket/REST futures blocked on a hung network call without honoring cancellation; runtime worker starvation under load; a spawned task panicking during shutdown; slow DNS or proxy stalls keeping futures alive past the timeout.","solutions":["Look at the full joined message (it may include the session-task error too) and the log lines from the failing spawned future for the root cause","Make spawned command futures cancellation-aware (tokio::select! on the group's cancellation token) so they exit promptly on begin_shutdown","Check network path (proxy/DNS) for hangs; add connect/IO timeouts to outbound calls so tasks cannot block indefinitely","If it recurs under load, reduce worker starvation (avoid blocking calls on tokio workers) and retry the connect"],"exampleFix":"// before: command future blocks on network call without observing shutdown\nasync move { ws.send(subscribe_msg).await?; Ok(()) }\n// after: bound the call and respect the shutdown token\nasync move {\n    tokio::select! {\n        _ = cancel.cancelled() => Ok(()),\n        r = tokio::time::timeout(Duration::from_secs(5), ws.send(subscribe_msg)) => r.map_err(Into::into),\n    }\n}","handlingStrategy":"try-catch","validationCode":"// Verify network reachability and timeouts before connect to reduce hung tasks:\ncurl -m 5 -sS https://api.binance.com/api/v3/ping && echo \"binance spot reachable\"","typeGuard":null,"tryCatchPattern":"// The teardown error surfaces from connect; catch and inspect the joined message:\nmatch client.connect().await {\n    Err(e) if e.to_string().contains(\"failed to finish Binance Spot data command tasks\") => {\n        log::warn!(\"partial-connect teardown timed out; safe to retry connect: {e:#}\");\n    }\n    r => r?,\n}","preventionTips":["Add explicit connect/IO timeouts on any custom WebSocket or REST futures spawned into the client","Ensure spawned tasks honor the group's cancellation token (tokio::select! on cancellation)","Keep the runtime unblocked: no std blocking sleeps/IO on tokio workers","Check proxy/DNS health so command futures cannot hang past the 2s shutdown window"],"tags":["websocket","shutdown","task-timeout","tokio","binance-spot"],"backgroundTag":"request-timeout","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}