What causes "Failed to start Derive task generation: {e}" in nautechsystems/nautilus_trader?

Recreate the client or ensure a clean disconnect before calling connect again.

Failed to start Derive task generation: {e}

Error message

Failed to start Derive task generation: {e}

What it means

Right after starting the session task generation, connect also calls pending_tasks.start_generation(). If the pending-task group cannot start a new generation, connect raises this message (after tearing down the partial connect). Note the session generation already succeeded at this point, so the client is half-initialized before teardown.

Source

Thrown at crates/adapters/derive/src/execution.rs:608

            && self.session_tasks.is_open()
            && self.pending_tasks.is_open()
        {
            return Ok(());
        }

        log::info!("Connecting Derive execution client");

        if self.cancellation_token.is_cancelled()
            || !self.session_tasks.is_open()
            || !self.pending_tasks.is_open()
        {
            self.teardown_partial_connect().await?;
            self.session_tasks
                .start_generation()
                .map_err(|e| anyhow::anyhow!("Failed to start Derive session generation: {e}"))?;
            self.pending_tasks
                .start_generation()
                .map_err(|e| anyhow::anyhow!("Failed to start Derive task generation: {e}"))?;
            self.cancellation_token = CancellationToken::new();
        }
        let cancellation_token = self.cancellation_token.clone();
        let ws_shutdown = self.ws_client.shutdown_handle();
        let setup_guard =
            TaskGroupGuard::new(&[&self.session_tasks, &self.pending_tasks], move || {
                cancellation_token.cancel();
                ws_shutdown.begin_shutdown();
            });

        self.ensure_instruments_initialized()
            .await
            .context("failed to initialize Derive instruments")?;

        self.ws_client
            .connect()
            .await
            .context("failed to connect Derive WebSocket")?;

View on GitHub (pinned to 18893faf8b)

Solutions

Recreate the client or ensure a clean disconnect before calling connect again.
Avoid concurrent connect invocations; serialize lifecycle transitions.
Check that prior connect failures fully tore down both task groups (teardown_partial_connect) before retrying.
Inspect the wrapped TaskGroup error for the exact pending_tasks state violation.

Defensive patterns

Strategy: try-catch

Validate before calling

if client.is_connected() || client.is_shutting_down() {
    return Err("connect not allowed in current state".into());
}

Try / catch

if let Err(e) = exec_client.connect().await {
    if e.to_string().contains("start Derive task generation") {
        // pending task group in bad state; teardown ran — recreate the client
    }
    return Err(e);
}

Prevention

Ensure prior failed connects fully tear down both task groups before retry
Never share one execution client across concurrent connect attempts
Prefer recreating the client over retrying connect after generation-start failures

When it happens

Trigger: connect called when the pending_tasks TaskGroup is already open or mid-shutdown; concurrent connect attempts; a prior failed connect that left pending_tasks open.

Common situations: Double-connect on the same instance; reconnect attempts without a full teardown; task group state carried over from a previously failed connection cycle.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/f37da087a3ced676. Report an issue: GitHub.