nautechsystems/nautilus_trader · error · anyhow::Error
Failed to terminate Polymarket data tasks: {e}
Error message
Failed to terminate Polymarket data tasks: {e} What it means
Raised by await_tasks_with_timeout when tasks.finish_shutdown(graceful_timeout, abort_timeout) fails to cleanly terminate all Polymarket data tasks within the computed graceful/abort windows during disconnect. Some data task (WS handler, instrument refresh, resolve poll) did not finish or could not be aborted as expected.
Source
Thrown at crates/adapters/polymarket/src/data/lifecycle.rs:426
}
};
self.tasks
.spawn(future)
.map_err(|e| anyhow::anyhow!("failed to register Polymarket resolve poll: {e}"))?;
Ok(())
}
pub(super) async fn await_tasks_with_timeout(
&self,
timeout: tokio::time::Duration,
) -> anyhow::Result<()> {
self.tasks.begin_shutdown();
let graceful_timeout = (timeout / 2).min(TASK_GRACEFUL_SHUTDOWN_TIMEOUT);
let abort_timeout = timeout.saturating_sub(graceful_timeout);
self.tasks
.finish_shutdown(graceful_timeout, abort_timeout)
.await
.map_err(|e| anyhow::anyhow!("Failed to terminate Polymarket data tasks: {e}"))?;
Ok(())
}
pub(super) fn start_client(&mut self) {
log::info!("Starting Polymarket data client: {}", self.client_id);
self.ensure_position_event_subscription();
}
pub(super) fn stop_client(&mut self) {
log::info!("Stopping Polymarket data client: {}", self.client_id);
self.tasks.begin_shutdown();
self.ws_client.begin_shutdown();
self.rtds_feed.begin_shutdown();
self.is_connected
.store(false, std::sync::atomic::Ordering::Relaxed);
self.clear_position_event_subscription();
}
View on GitHub (pinned to 18893faf8b)
Solutions
- Increase the timeout passed to disconnect() so graceful/abort windows are sufficient
- Check network reachability of Polymarket Gamma/WS endpoints that tasks may be blocked on
- Upgrade/patch if a task ignores cancellation tokens; ensure tasks honor the cancellation token
- Recreate the client if its task group is wedged; start_generation() runs on next connect
Example fix
// before client.disconnect().await?; // tiny default timeout // after client.disconnect_with_timeout(Duration::from_secs(30)).await?;
Defensive patterns
Strategy: try-catch
Validate before calling
// pick a generous disconnect timeout up front let timeout = Duration::from_secs(30); // not milliseconds assert!(timeout > Duration::from_secs(5));
Try / catch
if let Err(e) = client.disconnect_with_timeout(Duration::from_secs(30)).await {
log::error!("task termination incomplete: {e}; recreating client");
client = build_client(cfg)?; // wedged group: rebuild
} Prevention
- Pass a disconnect timeout large enough for HTTP-bound tasks (resolve poll) to observe cancellation
- Ensure tasks' network calls use timeouts so they cannot hang indefinitely
- Monitor endpoint latency to Polymarket Gamma API; slow endpoints lengthen shutdown
- On repeated shutdown timeouts, recreate the client instead of reusing the task group
When it happens
Trigger: Calling disconnect()/disconnect_client while a background task is stuck (e.g. blocking HTTP call in resolve poll or a wedged WS read) beyond half of the given timeout or TASK_GRACEFUL_SHUTDOWN_TIMEOUT, so finish_shutdown reports failure.
Common situations: Unresponsive network stalls in the resolve-poll HTTP client; instrument refresh blocked on a slow Gamma API call; very short user-supplied timeout values making graceful+abort windows too tight.
Understand the failure class
Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.
Related errors
- Failed to start Polymarket WebSocket handler task: {e}
- failed to register Polymarket instrument refresh: {e}
- Polymarket task admission is closed: {e}
- failed to register Polymarket resolve poll: {e}
- task slot is already occupied
AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08).
Data as JSON: /api/errors/a3f52ff99f9b4d9c.
Report an issue: GitHub.