nautechsystems/nautilus_trader · error

Failed to start Derive data task generation: {e}

Error message

Failed to start Derive data task generation: {e}

What it means

Right after starting the session task generation, the Derive data client's `connect` also starts a new generation for `pending_tasks`. This error wraps a failure of that second generation start; the client cannot queue pending subscription tasks for the new session.

Source

Thrown at crates/adapters/derive/src/data.rs:749

            && self.session_tasks.is_open()
            && self.pending_tasks.is_open()
        {
            return Ok(());
        }

        // Completes the async teardown deferred by sync reset()/stop().
        if self.cancellation_token.is_cancelled()
            || !self.session_tasks.is_open()
            || !self.pending_tasks.is_open()
        {
            self.teardown_partial_connect().await?;
            self.cancellation_token = CancellationToken::new();
            self.session_tasks.start_generation().map_err(|e| {
                anyhow::anyhow!("Failed to start Derive data session generation: {e}")
            })?;
            self.pending_tasks
                .start_generation()
                .map_err(|e| anyhow::anyhow!("Failed to start Derive data task generation: {e}"))?;
        }
        let cancellation_token = self.cancellation_token.clone();
        let ws_shutdown = self.ws_client.shutdown_handle();
        let setup_guard =
            TaskGroupGuard::new(&[&self.session_tasks, &self.pending_tasks], move || {
                cancellation_token.cancel();
                ws_shutdown.begin_shutdown();
            });

        if !self.config.currencies.is_empty() {
            self.provider
                .load_all(None)
                .await
                .context("failed to load Derive instruments")?;
            self.cache_provider_instruments();
        }

        self.ws_client

View on GitHub (pinned to 18893faf8b)

Solutions

1. Inspect the wrapped `{e}` for the exact generation-state error.
2. Ensure the prior connect/disconnect fully completed before retrying.
3. Serialize lifecycle calls (connect/disconnect) on the data client.
4. Rebuild the data client if the pending task group is shut down permanently.
5. Verify teardown_partial_connect ran (or its error was propagated) after any failed connect.

Example fix

// before
if let Err(e) = client.connect().await { log(e); } // then retry immediately
client.connect().await?;
// after
if let Err(e) = client.connect().await { client.disconnect().await?; } // clean state
client.connect().await?;

// Clean state before connect:
// data_client.disconnect().await?; // ensures pending/session groups reset

if let Err(e) = data_client.connect().await {
    if e.to_string().contains("task generation") {
        data_client.disconnect().await.ok();
        tokio::time::sleep(BACKOFF).await;
        data_client.connect().await?;
    }
}

When it happens

Trigger: Calling `connect()` when `pending_tasks` is not open/empty and `start_generation()` returns Err — group closed, invalid generation state, or partial teardown left it unusable.

Common situations: Reconnect cycles where a previous pending-task generation never finished; concurrent connect attempts; a failed earlier connect leaving pending_tasks in a bad state.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Failed to start Derive data session generation: {e}
• failed to start WebSocket handler task generation: {e}
• Derive WebSocket event receiver not initialized
• joined shutdown errors (std::mem::take(&mut self.shutdown_er
• invalid Derive subscription channel `{channel}`

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/6bfa69202e9ef4b8. Report an issue: GitHub.