What causes "Failed to start recoverable execution transition: {e}" in nautechsystems/nautilus_trader?

Verify database health and connectivity, then retry the release

nautechsystems/nautilus_trader · error · anyhow::Error

Failed to start recoverable execution transition: {e}

Error message

Failed to start recoverable execution transition: {e}

What it means

Thrown when mark_execution_intent_recoverable fails at pool.begin(), before the intent is locked or inspected. The wrapped sqlx error means a connection could not be acquired (database unreachable, pool exhausted, dropped connection). No state changed when this fires.

Source

Thrown at crates/adapters/blockchain/src/cache/database.rs:3431

        .bind(nonce_db)
        .execute(&self.pool)
        .await
        .map_err(|e| anyhow::anyhow!("Failed to assign nonce {nonce} to execution intent: {e}"))?;
        anyhow::ensure!(
            result.rows_affected() == 1,
            "Execution intent {intent_id} is not prepared for nonce {nonce}"
        );
        Ok(())
    }

    /// Releases an intent when no broadcast attempt can have occurred.
    ///
    /// # Errors
    ///
    /// Returns an error if the intent advanced to broadcast or persistence fails.
    pub async fn mark_execution_intent_recoverable(&self, intent_id: i64) -> anyhow::Result<()> {
        let mut transaction = self.pool.begin().await.map_err(|e| {
            anyhow::anyhow!("Failed to start recoverable execution transition: {e}")
        })?;
        let current_status = sqlx::query_scalar::<_, String>(
            "SELECT status FROM execution_intent WHERE id = $1 FOR UPDATE",
        )
        .bind(intent_id)
        .fetch_optional(&mut *transaction)
        .await
        .map_err(|e| anyhow::anyhow!("Failed to lock recoverable execution intent: {e}"))?
        .ok_or_else(|| anyhow::anyhow!("Execution intent {intent_id} was not found"))?;
        anyhow::ensure!(
            matches!(current_status.as_str(), "prepared" | "signed"),
            "Execution intent {intent_id} is {current_status}, not recoverable before broadcast"
        );
        let result = sqlx::query(
            "
            UPDATE execution_intent
            SET status = 'recoverable', active = FALSE, updated_at = NOW()
            WHERE id = $1 AND status IN ('prepared', 'signed')

View on GitHub (pinned to 2114cf6f76)

Solutions

Verify database health and connectivity, then retry the release
Increase pool max_connections/acquire_timeout for peak concurrency
Schedule recovery sweeps outside peak activity windows

Defensive patterns

Strategy: retry

Try / catch

match db.mark_execution_intent_recoverable(intent_id).await {
    Err(e) if e.downcast_ref::<sqlx::Error>().is_some_and(|se| matches!(se, sqlx::Error::PoolTimedOut | sqlx::Error::Io(_))) => {
        // retry after pool pressure subsides; nothing changed on failure
    }
    other => other?,
}

Prevention

Run recovery sweeps with bounded concurrency against the same pool
Verify pool health before recovery passes that release many intents
Keep recovery logic idempotent so retries are always safe

When it happens

Trigger: Releasing a pre-broadcast intent while the pool is fully consumed by other database work; database restart mid-operation; connection acquisition timing out under burst load.

Common situations: Load spikes exhausting the pool; failover of the database tier; connectivity blips.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@2114cf6f76 (2026-08-21). Data as JSON: /api/errors/6f128a0caefcf50e. Report an issue: GitHub.