nautechsystems/nautilus_trader · error

Failed to mark execution intent recoverable

Error message

Failed to mark execution intent recoverable: {e}

What it means

The UPDATE that flips execution_intent from 'prepared' to 'recoverable' failed at the database level; the underlying sqlx error is wrapped as "Failed to mark execution intent recoverable: {e}". This is a persistence failure, not a business-rule rejection (rows_affected is checked separately).

Solutions

  1. Inspect the wrapped sqlx error source for the root cause (deadlock, connection closed, timeout) via the anyhow chain.
  2. Retry the whole mark_execution_intent_recoverable call with backoff — the failed transaction rolled back, so the state is unchanged and safe to redo.
  3. Check database health: connection limits, max_connections, statement_timeout, and recent deadlock logs.
  4. Reduce time spent holding the transaction open (avoid slow work between begin and commit) to lower deadlock/timeout risk.
Defensive patterns

Strategy: retry

Try / catch

// Retry transient DB failures with backoff; transaction rolled back so state is safe
for attempt in 0..3 {
    match db.mark_execution_intent_recoverable(id).await {
        Ok(()) => break,
        Err(e) if attempt < 2 && is_transient_db_error(&e) => tokio::time::sleep(backoff(attempt)).await,
        Err(e) => return Err(e),
    }
}

Prevention

When it happens

Trigger: The `UPDATE execution_intent SET status='recoverable' ...` statement inside the open transaction fails — connection drop mid-transaction, deadlock on the FOR UPDATE-locked row, statement timeout, or constraint/permission failure.

Common situations: Database connection pool exhausted or idle-timeout closed the connection while the transaction was held open; another long transaction deadlocked on the intent row; Postgres restart/failover during recovery; network partition between the adapter and the database.

Related errors


AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/eb7c2244f4886398. Report an issue: GitHub.

Appendix: source

Thrown at crates/adapters/blockchain/src/cache/database.rs:6536

        .fetch_optional(&mut *transaction)
        .await
        .map_err(|e| anyhow::anyhow!("Failed to lock recoverable execution intent: {e}"))?
        .ok_or_else(|| anyhow::anyhow!("Execution intent {intent_id} was not found"))?;
        anyhow::ensure!(
            current_status == "prepared",
            "Execution intent {intent_id} is {current_status}, not recoverable before signing"
        );
        let result = sqlx::query(
            "
            UPDATE execution_intent
            SET status = 'recoverable', active = FALSE, updated_at = NOW()
            WHERE id = $1 AND status = 'prepared'
            ",
        )
        .bind(intent_id)
        .execute(&mut *transaction)
        .await
        .map_err(|e| anyhow::anyhow!("Failed to mark execution intent recoverable: {e}"))?;
        anyhow::ensure!(
            result.rows_affected() == 1,
            "Execution intent {intent_id} is not recoverable from preparation"
        );
        sqlx::query(
            "
            INSERT INTO execution_transaction_transition (
                intent_id, transition_key, from_status, to_status
            ) VALUES ($1, 'recoverable', $2, 'recoverable')
            ON CONFLICT (intent_id, transition_key) DO NOTHING
            ",
        )
        .bind(intent_id)
        .bind(current_status)
        .execute(&mut *transaction)
        .await
        .map_err(|e| anyhow::anyhow!("Failed to record recoverable transition: {e}"))?;
        transaction

View on GitHub (pinned to 18893faf8b)