nautechsystems/nautilus_trader · error · anyhow::Error

Failed to start execution status transition

Error message

Failed to start execution status transition: {e}

What it means

This error wraps a failure to BEGIN a SQL transaction via sqlx's pool when starting an execution-status transition in the blockchain cache database. The transaction is needed to row-lock (SELECT ... FOR UPDATE) the execution_intent row and apply the state change atomically. If the pool cannot start a transaction (connection unavailable, pool exhausted, database down), the transition is aborted with this message.

Solutions

  1. Verify the database is reachable and credentials are correct (psql with the same DATABASE_URL).
  2. Check pool sizing (max_connections, acquire_timeout) versus concurrent load; increase limits or reduce concurrency.
  3. Retry the operation with backoff — sqlx pool acquisition failures are often transient.
  4. Inspect the inner `{e}` message for the root sqlx error (connection refused vs timeout vs auth).

Example fix

// before
let mut transaction = self.pool.begin().await.map_err(|e| anyhow::anyhow!("Failed to start execution status transition: {e}"))?;
// after
let mut transaction = self.pool
    .begin()
    .await
    .with_context(|| format!("Failed to start execution status transition (db={})", self.pool.size()))?;
Defensive patterns

Strategy: retry

Validate before calling

// Rust: check pool health before attempting the transition
if self.pool.is_closed() {
    anyhow::bail!("database pool closed");
}
// optionally: let _ = sqlx::query("SELECT 1").execute(&self.pool).await?;

Try / catch

match db.record_execution_status(&intent_id, status, ...).await {
    Ok(()) => {},
    Err(e) if e.to_string().contains("Failed to start execution status transition") => {
        tracing::warn!(error = %e, "transient db error; retrying with backoff");
        tokio::time::sleep(backoff.next()).await;
    }
    Err(e) => return Err(e),
}

Prevention

When it happens

Trigger: Calling the execution-status transition method (e.g. record_execution_status/transition) when DatabasePool::begin() fails: connection pool exhausted (max_connections in use), database unreachable, connection dropped mid-handshake, or TLS/auth failure at connect time.

Common situations: Database restarted or failover in progress while the adapter processes receipts; pool saturation under high event throughput; wrong DATABASE_URL / credentials after a config change; network partition between the adapter and Postgres; idle connections reaped by a firewall/load balancer.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/70822522b3c9caf0. Report an issue: GitHub.

Appendix: source

Thrown at crates/adapters/blockchain/src/cache/database.rs:6783

            .map(i64::try_from)
            .transpose()
            .with_context(|| {
                format!(
                    "Execution block number {} exceeds PostgreSQL BIGINT",
                    block_number.unwrap_or_default()
                )
            })?;
        let gas_used_db = gas_used.map(i64::try_from).transpose().with_context(|| {
            format!(
                "Execution gas used {} exceeds PostgreSQL BIGINT",
                gas_used.unwrap_or_default()
            )
        })?;
        let mut transaction = self
            .pool
            .begin()
            .await
            .map_err(|e| anyhow::anyhow!("Failed to start execution status transition: {e}"))?;
        let (current_status, fill_emitted, terminal_emitted) =
            sqlx::query_as::<_, (String, bool, bool)>(
            "SELECT status, fill_emitted, terminal_emitted FROM execution_intent WHERE id = $1 FOR UPDATE",
        )
        .bind(intent_id)
        .fetch_optional(&mut *transaction)
        .await
        .map_err(|e| anyhow::anyhow!("Failed to lock execution intent {intent_id}: {e}"))?
        .ok_or_else(|| anyhow::anyhow!("Execution intent {intent_id} was not found"))?;
        anyhow::ensure!(
            execution_transition_allowed(&current_status, status),
            "Invalid execution transition for intent {intent_id}: {current_status} -> {}",
            status.as_str()
        );

        let active = match status {
            TransactionStatus::Finalized | TransactionStatus::Reverted => {
                !fill_emitted && !terminal_emitted

View on GitHub (pinned to 18893faf8b)