nautechsystems/nautilus_trader · warning · anyhow::Error

Failed to terminate Hyperliquid data session tasks: {e}

Error message

Failed to terminate Hyperliquid data session tasks: {e}

What it means

This error wraps a failure from the data client's session task shutdown in `await_session_tasks` (crates/adapters/hyperliquid/src/data.rs:270). During teardown the client asks its managed WebSocket session tasks to stop, giving them 1 second to begin shutdown and 2 seconds to finish; if any task refuses to end in time or panics, `finish_shutdown` returns an error which is re-raised with this message. It indicates stale Hyperliquid market-data connections/tasks could not be cleanly terminated.

Source

Thrown at crates/adapters/hyperliquid/src/data.rs:270

        Ok(())
    }

    async fn await_pending_tasks(&self) -> anyhow::Result<()> {
        self.pending_tasks.begin_shutdown();
        self.pending_tasks
            .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2))
            .await
            .map_err(|e| anyhow::anyhow!("Failed to terminate Hyperliquid data tasks: {e}"))?;
        Ok(())
    }

    async fn await_session_tasks(&self) -> anyhow::Result<()> {
        self.session_tasks.begin_shutdown();
        self.session_tasks
            .finish_shutdown(Duration::from_secs(1), Duration::from_secs(2))
            .await
            .map_err(|e| {
                anyhow::anyhow!("Failed to terminate Hyperliquid data session tasks: {e}")
            })?;
        Ok(())
    }

    fn clear_stream_health(&self) {
        self.stream_health.lock().clear();
    }

    fn register_stream_health(&self, channel: MarketDataChannel, instrument_id: InstrumentId) {
        if !self.stream_health_monitor_enabled() {
            return;
        }

        self.stream_health
            .lock()
            .subscribe(channel, instrument_id, Instant::now());
    }

View on GitHub (pinned to 18893faf8b)

Solutions

1. Retry the disconnect after a short pause; the tasks usually finish or are abandoned by the timeout.
2. Check network/firewall conditions that keep the WebSocket from closing promptly.
3. If it reproduces, inspect custom session task code or proxy settings that block socket close; restart the process/node to clear stuck tasks.
4. Ensure you are on a recent adapter version where cancellation tokens are applied to all session tasks.

Example fix

// before
client.disconnect().await.expect("disconnect must succeed");
// after
if let Err(e) = client.disconnect().await {
    tracing::warn!("hyperliquid data teardown issue: {e:#}"); // tolerate timeout during shutdown
}

// Rust
match client.disconnect().await {
    Ok(()) => {},
    Err(e) if e.to_string().contains("Failed to terminate") => {
        tracing::warn!("data session tasks did not shut down in time: {e:#}");
        // safe to proceed/reconnect; tasks are force-aborted by timeout
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling teardown_partial_connect (via DataClient disconnect) when a spawned session task is blocked (e.g. stuck in a network read that ignores the cancellation token), hung, or takes longer than the 1s/2s shutdown timeouts.

Common situations: Network partitions or proxies holding the WebSocket open; a WS task blocked on a slow recv; system suspend/resume leaving sockets half-dead; calling disconnect while a large subscribe/reconnect loop is in flight.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• Hyperliquid WebSocket handler did not stop after abort
• Architect AX data WebSocket handler did not stop after abort
• Failed to terminate Hyperliquid data tasks: {e}
• Hyperliquid WebSocket handler failed: {error}
• Lighter WebSocket handler did not stop after abort

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/f869702d4b5d0c9b. Report an issue: GitHub.