nautechsystems/nautilus_trader · warning

disconnect timeout while disconnecting clients

Error message

disconnect timeout while disconnecting clients

What it means

finalize_stop disconnects all kernel clients under config.timeout_disconnection. If disconnect_clients() does not finish within that window, the timeout elapses, this error is constructed, and it is logged as "Error disconnecting clients" while the stop proceeds. It indicates a client hung during disconnect, but the node still continues its shutdown sequence.

Source

Thrown at crates/live/src/node/mod.rs:2363

            log::error!("Error stopping plug-in controllers: {e}");
        }
        self.kernel.stop_trader();
        let delay = self.kernel.delay_post_stop();
        log::info!("Awaiting residual events ({delay:?})...");

        self.shutdown_deadline = Some(dst::time::Instant::now() + delay);
        self.handle.set_shutting_down();
    }

    async fn finalize_stop(&mut self) -> anyhow::Result<()> {
        self.close_external_ingress();

        let timeout = self.config.timeout_disconnection;
        let deadline = dst::time::Instant::now() + timeout;
        let disconnect_result =
            match dst::time::timeout(timeout, self.kernel.disconnect_clients()).await {
                Ok(result) => result,
                Err(_) => Err(anyhow::anyhow!(
                    "disconnect timeout while disconnecting clients"
                )),
            };

        if let Err(ref e) = disconnect_result {
            log::error!("Error disconnecting clients: {e}");
        }

        let readiness_result = self.await_engines_disconnected(deadline).await;
        let kernel_result = self.kernel.finalize_stop().await;

        self.handle.set_stopped();

        let mut errors = Vec::new();
        if let Err(e) = disconnect_result {
            errors.push(e.to_string());
        }

View on GitHub (pinned to 18893faf8b)

Solutions

1. Increase timeout_disconnection in the live config to accommodate slow client teardown.
2. Check adapter logs for a specific client hanging in disconnect and fix or update that adapter.
3. Force-close stuck network connections (firewall reset, process restart) if a half-open socket is the cause.
4. This is logged but non-fatal; confirm all clients actually reconnected cleanly on the next startup.

Example fix

// before
timeout_disconnection: Duration::from_secs(2)
// after
timeout_disconnection: Duration::from_secs(30)

// Ensure disconnect budget is realistic for adapter teardown
assert!(config.timeout_disconnection >= Duration::from_secs(10));

// The node logs and continues, but callers of stop() should still check:
node.stop().await; // timeout is logged internally as "Error disconnecting clients"
// optionally verify state afterwards:
if node.is_running() { /* force cleanup or restart */ }

When it happens

Trigger: In finalize_stop (called from stop and all abort paths), dst::time::timeout(timeout_disconnection, disconnect_clients()) returns Err (mod.rs:2358-2368) because one or more clients' disconnect futures hang.

Common situations: Broken WebSocket connections whose adapters wait on a send/flush that can never complete; venue APIs unresponsive during disconnect; very small timeout_disconnection values; network partitions at shutdown time.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Betfair data shutdown failed: {}
• Binance Futures data teardown failed: {}
• Binance Futures shutdown failed: {}
• Bybit execution shutdown failed: {}
• subscription confirmation failed: {e}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/50f963314961f78f. Report an issue: GitHub.