nautechsystems/nautilus_trader · error · anyhow::Error

Failed to advance canonical nonce ledger: {e}

Error message

Failed to advance canonical nonce ledger: {e}

What it means

This error wraps SQLx failures from the UPDATE of execution_verification_nonce, which advances the wallet's next_canonical_nonce with an optimistic revision check inside the finality transaction (database.rs:7191). The nonce ledger must be advanced for the finality to commit, so any query failure aborts the transaction and is surfaced with this message.

Source

Thrown at crates/adapters/blockchain/src/cache/database.rs:7191

        .await
        .map_err(|e| anyhow::anyhow!("Failed to record verified finality transition: {e}"))?;

        let nonce_result = sqlx::query(
            "
            UPDATE execution_verification_nonce
            SET next_canonical_nonce = $3, revision = revision + 1, updated_at = NOW()
            WHERE chain_id = $1 AND wallet_address = $2
              AND next_canonical_nonce = $4 AND revision = $5
            ",
        )
        .bind(chain_id)
        .bind(finality.wallet_address)
        .bind(next_nonce)
        .bind(nonce)
        .bind(revision)
        .execute(&mut *transaction)
        .await
        .map_err(|e| anyhow::anyhow!("Failed to advance canonical nonce ledger: {e}"))?;
        anyhow::ensure!(
            nonce_result.rows_affected() == 1,
            "Canonical nonce ledger changed during finality transition"
        );
        transaction
            .commit()
            .await
            .map_err(|e| anyhow::anyhow!("Failed to commit verified finality transition: {e}"))?;
        Ok(())
    }

    /// Loads one durable execution intent by ID.
    pub(crate) async fn get_execution_intent(
        &self,
        intent_id: i64,
    ) -> anyhow::Result<ExecutionIntentRow> {
        sqlx::query_as::<_, ExecutionIntentRow>(
            "

View on GitHub (pinned to 18893faf8b)

Solutions

1. Inspect the wrapped {e} for the driver-level cause (connection reset, undefined table, type mismatch).
2. Apply pending database migrations to ensure execution_verification_nonce and its columns exist.
3. Verify the chain_id and wallet_address values passed in match the stored ledger row's types/format (checksummed vs lowercase address).
4. Retry the entire finality transition; it is transactional, so a failed nonce update rolls back cleanly and a retry with the current revision is safe.
5. Check pool configuration and statement timeouts if failures correlate with load.

Example fix

// before: retry only the nonce update out of band
let _ = db.advance_nonce(chain_id, wallet).await;
// after: retry the whole transactional transition so state stays consistent
for attempt in 0..3 {
    match db.record_verified_finality(&finality).await {
        Ok(()) => break,
        Err(e) if attempt < 2 && is_transient(&e) => tokio::time::sleep(backoff(attempt)).await,
        Err(e) => return Err(e.context("advance canonical nonce ledger")),
    }
}

let ledger: Option<(i64, i64)> = sqlx::query_as(
    "SELECT next_canonical_nonce, revision FROM execution_verification_nonce WHERE chain_id = $1 AND wallet_address = $2")
    .bind(chain_id).bind(wallet).fetch_optional(&pool).await?;
anyhow::ensure!(ledger.is_some(), "nonce ledger row missing for {wallet} on chain {chain_id}");

fn is_retryable_db_error(e: &anyhow::Error) -> bool {
    let s = e.to_string();
    s.contains("connection") || s.contains("timed out") || s.contains("reset by peer")
}

match res {
    Err(e) if is_retryable_db_error(&e) => retry_with_backoff(|| db.record_verified_finality(&finality)).await?,
    Err(e) => return Err(e.context("nonce ledger update failed")),
    Ok(()) => {}
}

When it happens

Trigger: The UPDATE ... SET next_canonical_nonce = $3 ... WHERE chain_id = $1 AND wallet_address = $2 AND next_canonical_nonce = $4 AND revision = $5 fails at the driver level: connection loss, statement timeout, wrong parameter types (e.g. wallet_address encoding vs column type), or the table/columns missing due to unapplied migrations.

Common situations: Postgres briefly unavailable or restarted during a finality commit; migrations not applied so execution_verification_nonce does not exist; parameter type mismatch after a chain_id/wallet column type change; network flakiness between the adapter and the database under load.

Related errors

• Failed to update {event_family} pool event-family checkpoint
• Failed to finalize pool event sync progress: {e}
• Failed to lock execution intent for nonce assignment: {e}
• Failed to persist pre-sign verification: {e}
• Failed to commit verified action evidence: {e}

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/49ba3fb29be41d02. Report an issue: GitHub.