nautechsystems/nautilus_trader · error

Failed to terminate Tardis tasks: {e}

Error message

Failed to terminate Tardis tasks: {e}

What it means

Raised in connect when finish_shutdown on the Tardis task set fails (does not complete within the 1s/2s grace windows) while preparing for a (re)connection. The prior generation of stream tasks could not be terminated cleanly, so connect aborts.

Source

Thrown at crates/adapters/tardis/src/data.rs:518

    fn unsubscribe_funding_rates(&mut self, _cmd: &UnsubscribeFundingRates) -> anyhow::Result<()> {
        Ok(())
    }

    async fn connect(&mut self) -> anyhow::Result<()> {
        if self.is_connected() && self.tasks.is_open() {
            return Ok(());
        }

        if self.config.options.is_empty() && self.config.stream_options.is_empty() {
            anyhow::bail!("Either replay `options` or `stream_options` must be provided");
        }

        if !self.tasks.is_open() {
            self.tasks
                .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2))
                .await
                .map_err(|e| anyhow::anyhow!("Failed to terminate Tardis tasks: {e}"))?;
            self.tasks
                .start_generation()
                .map_err(|e| anyhow::anyhow!("Failed to start Tardis task generation: {e}"))?;
            self.cancellation_token = self.tasks.cancellation_token();
        }

        let is_stream_mode = self.is_stream_mode();
        let book_snapshot_output = self.config.book_snapshot_output.clone();
        let extract_bbo_as_quotes = self.config.extract_bbo_as_quotes;

        let http_client = TardisHttpClient::new(
            self.config
                .api_key
                .as_ref()
                .map(|value| value.expose_secret()),
            None,
            None,
            self.config.normalize_symbols,

View on GitHub (pinned to 18893faf8b)

Solutions

1. Ensure the WS task loop selects on the cancellation token so it exits promptly.
2. Increase the finish_shutdown grace durations if tasks legitimately need longer to stop.
3. Check network conditions toward the Tardis Machine; a hung socket can stall graceful shutdown.
4. Drop/recreate the client if the task set is wedged.

Example fix

// before
.map_err(|e| anyhow::anyhow!("Failed to terminate Tardis tasks: {e}"))?;
// after: make the ws loop cancellation-responsive
// tokio::select! { _ = cancellation_token.cancelled() => break, msg = ws.next() => ... }

match connect().await {
    Err(e) if e.to_string().contains("Failed to terminate Tardis tasks") => {
        log::warn!("previous tasks wedged; restarting client");
        client = TardisDataClient::new(config)?; // recreate instead of retrying in-place
        client.connect().await?;
    }
    other => other?,
}

When it happens

Trigger: Calling connect while existing WebSocket tasks ignore the cancellation token and keep running past the finish_shutdown timeouts.

Common situations: WebSocket read loop blocked on a stalled Tardis Machine connection without honoring cancellation; very slow network teardown; tasks awaiting a long I/O future between cancellation checks.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• {context}: {}
• Failed to terminate Lighter tasks: {}
• failed to finish Binance Futures data command tasks: {e}
• Binance Futures data teardown failed: {}
• Binance Futures shutdown failed: {}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/6ba8bf3c79c486b4. Report an issue: GitHub.