What causes "Failed to start Derive session generation: {e}" in nautechsystems/nautilus_trader?

Ensure connect is called at most once per client lifecycle; recreate the client instead of reconnecting manually.

Failed to start Derive session generation: {e}

Error message

Failed to start Derive session generation: {e}

What it means

connect starts a new generation on both the session and pending TaskGroups before spawning work. If session_tasks.start_generation() fails (group already started or in an invalid state), connect tears down the partial connect and raises this message. It means the Derive execution client could not initialize its session task group for a new connection generation.

Source

Thrown at crates/adapters/derive/src/execution.rs:605

    async fn connect(&mut self) -> anyhow::Result<()> {
        if self.is_connected()
            && !self.cancellation_token.is_cancelled()
            && self.session_tasks.is_open()
            && self.pending_tasks.is_open()
        {
            return Ok(());
        }

        log::info!("Connecting Derive execution client");

        if self.cancellation_token.is_cancelled()
            || !self.session_tasks.is_open()
            || !self.pending_tasks.is_open()
        {
            self.teardown_partial_connect().await?;
            self.session_tasks
                .start_generation()
                .map_err(|e| anyhow::anyhow!("Failed to start Derive session generation: {e}"))?;
            self.pending_tasks
                .start_generation()
                .map_err(|e| anyhow::anyhow!("Failed to start Derive task generation: {e}"))?;
            self.cancellation_token = CancellationToken::new();
        }
        let cancellation_token = self.cancellation_token.clone();
        let ws_shutdown = self.ws_client.shutdown_handle();
        let setup_guard =
            TaskGroupGuard::new(&[&self.session_tasks, &self.pending_tasks], move || {
                cancellation_token.cancel();
                ws_shutdown.begin_shutdown();
            });

        self.ensure_instruments_initialized()
            .await
            .context("failed to initialize Derive instruments")?;

        self.ws_client

View on GitHub (pinned to 18893faf8b)

Solutions

Ensure connect is called at most once per client lifecycle; recreate the client instead of reconnecting manually.
Synchronize connect calls (single owner/task) to avoid concurrent start_generation on the same group.
The error path already runs teardown_partial_connect — check its output for the underlying state; retry after a clean teardown.
Verify the wrapped TaskGroup error to confirm whether the group was already started.

Defensive patterns

Strategy: try-catch

Validate before calling

// Only connect a fresh or cleanly torn-down client:
if client.is_connected() { return Ok(()); }

Try / catch

if let Err(e) = exec_client.connect().await {
    if e.to_string().contains("start Derive session generation") {
        // client half-initialized; teardown already ran — recreate client
    }
    return Err(e);
}

Prevention

Avoid double-connect; guard with an is_connected check or connection state machine
Do not attempt manual reconnect on the same instance; instantiate a new client
Keep lifecycle transitions single-threaded/serialized

When it happens

Trigger: Calling connect when session_tasks is already open or mid-shutdown; double-connect on the same client instance; connect invoked concurrently from two places.

Common situations: Manual reconnect logic calling connect twice; racing connect calls from different actors/tasks; calling connect on a client that was never fully torn down after a previous failure.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/6d5ecae37210d052. Report an issue: GitHub.