nautechsystems/nautilus_trader · error · anyhow::Error

Failed to terminate Polymarket data tasks

Error message

Failed to terminate Polymarket data tasks: {e}

What it means

Raised by await_tasks_with_timeout when tasks.finish_shutdown(graceful_timeout, abort_timeout) fails to cleanly terminate all Polymarket data tasks within the computed graceful/abort windows during disconnect. Some data task (WS handler, instrument refresh, resolve poll) did not finish or could not be aborted as expected.

Solutions

  1. Increase the timeout passed to disconnect() so graceful/abort windows are sufficient
  2. Check network reachability of Polymarket Gamma/WS endpoints that tasks may be blocked on
  3. Upgrade/patch if a task ignores cancellation tokens; ensure tasks honor the cancellation token
  4. Recreate the client if its task group is wedged; start_generation() runs on next connect

Example fix

// before
client.disconnect().await?; // tiny default timeout
// after
client.disconnect_with_timeout(Duration::from_secs(30)).await?;
Defensive patterns

Strategy: try-catch

Validate before calling

// pick a generous disconnect timeout up front
let timeout = Duration::from_secs(30); // not milliseconds
assert!(timeout > Duration::from_secs(5));

Try / catch

if let Err(e) = client.disconnect_with_timeout(Duration::from_secs(30)).await {
    log::error!("task termination incomplete: {e}; recreating client");
    client = build_client(cfg)?; // wedged group: rebuild
}

Prevention

When it happens

Trigger: Calling disconnect()/disconnect_client while a background task is stuck (e.g. blocking HTTP call in resolve poll or a wedged WS read) beyond half of the given timeout or TASK_GRACEFUL_SHUTDOWN_TIMEOUT, so finish_shutdown reports failure.

Common situations: Unresponsive network stalls in the resolve-poll HTTP client; instrument refresh blocked on a slow Gamma API call; very short user-supplied timeout values making graceful+abort windows too tight.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors


AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/a3f52ff99f9b4d9c. Report an issue: GitHub.

Appendix: source

Thrown at crates/adapters/polymarket/src/data/lifecycle.rs:426

            }
        };
        self.tasks
            .spawn(future)
            .map_err(|e| anyhow::anyhow!("failed to register Polymarket resolve poll: {e}"))?;
        Ok(())
    }

    pub(super) async fn await_tasks_with_timeout(
        &self,
        timeout: tokio::time::Duration,
    ) -> anyhow::Result<()> {
        self.tasks.begin_shutdown();
        let graceful_timeout = (timeout / 2).min(TASK_GRACEFUL_SHUTDOWN_TIMEOUT);
        let abort_timeout = timeout.saturating_sub(graceful_timeout);
        self.tasks
            .finish_shutdown(graceful_timeout, abort_timeout)
            .await
            .map_err(|e| anyhow::anyhow!("Failed to terminate Polymarket data tasks: {e}"))?;
        Ok(())
    }

    pub(super) fn start_client(&mut self) {
        log::info!("Starting Polymarket data client: {}", self.client_id);
        self.ensure_position_event_subscription();
    }

    pub(super) fn stop_client(&mut self) {
        log::info!("Stopping Polymarket data client: {}", self.client_id);
        self.tasks.begin_shutdown();
        self.ws_client.begin_shutdown();
        self.rtds_feed.begin_shutdown();
        self.is_connected
            .store(false, std::sync::atomic::Ordering::Relaxed);
        self.clear_position_event_subscription();
    }

View on GitHub (pinned to 18893faf8b)