nautechsystems/nautilus_trader · error

Failed to start market shard forwarder: {e}; startup rollbac

Error message

Failed to start market shard forwarder: {e}; startup rollback failed: {close_error}

What it means

After opening a new shard, connect_new_shard spawns a message forwarder task that pumps shard messages into the merged pool stream. If spawning the forwarder fails ({e}), the freshly opened shard is closed as a rollback; if that close also fails ({close_error}), both errors are combined into this message; otherwise the single forwarder error is raised.

Source

Thrown at crates/adapters/polymarket/src/websocket/pool.rs:672

        client.connect().await?;

        let handle = client.clone_subscription_handle();
        let rx = client
            .take_message_receiver()
            .ok_or_else(|| anyhow::anyhow!("Market shard receiver unavailable after connect"))?;
        let forwarder = match self.spawn_forwarder(rx, is_primary) {
            Ok(forwarder) => forwarder,
            Err((e, forwarder)) => {
                let shard = Box::new(ShardEntry {
                    client,
                    handle,
                    forwarder,
                    owned: 0,
                    closing: true,
                });

                if let Err(close_error) = self.close_shard(id, shard).await {
                    anyhow::bail!(
                        "Failed to start market shard forwarder: {e}; startup rollback failed: \
                         {close_error}"
                    );
                }
                anyhow::bail!("Failed to start market shard forwarder: {e}");
            }
        };

        let shard = ShardEntry {
            client,
            handle,
            forwarder,
            owned: 0,
            closing: false,
        };
        let rejected_shard = {
            let mut state = self.state.lock();

View on GitHub (pinned to 18893faf8b)

Solutions

  1. Inspect {e} for the forwarder root cause (often runtime shutdown — check the tokio runtime is alive)
  2. If {close_error} indicates a dead connection, cleanup can be ignored; the shard never entered service
  3. Retry the subscription after confirming the runtime/runtime handle is valid
  4. Avoid opening shards during application shutdown; tear down the pool instead
Defensive patterns

Strategy: retry

Validate before calling

// ensure a live runtime before shard expansion
assert!(!tokio::runtime::Handle::try_current().is_err(), "no runtime");

Try / catch

if let Err(e) = pool.subscribe_one(asset_id).await {
    if e.to_string().contains("forwarder") {
        log::warn!("shard forwarder failed: {e}");
        // retry after confirming runtime health
    }
}

Prevention

When it happens

Trigger: Forwarder task spawn/start fails (e.g., tokio spawn failure, runtime shutting down, channel setup failure) on a newly connected shard; rollback close of that shard also errors.

Common situations: Runtime shutdown while the pool is expanding capacity; resource exhaustion (task/channel limits); connection established but the message-pumping infrastructure failed to start.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors


AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/e4806eccf1dc1590. Report an issue: GitHub.