nautechsystems/nautilus_trader · error · anyhow::Error

Finalized nonce {} does not match canonical nonce {stored_no

Error message

Finalized nonce {} does not match canonical nonce {stored_nonce}

What it means

Thrown by `record_execution_finality_verified` when the nonce of the finalized transaction does not equal the ledger's stored next-canonical nonce (execution_verification_nonce.next_canonical_nonce). The function requires finality.nonce to be consumed exactly in canonical order; any divergence aborts the transaction. This protects against double-spending nonce gaps or recording finality out of order.

Source

Thrown at crates/adapters/blockchain/src/cache/database.rs:6953

                "
                SELECT manifest_version, manifest_digest, next_canonical_nonce, revision
                FROM execution_verification_nonce
                WHERE chain_id = $1 AND wallet_address = $2
                FOR UPDATE
                ",
            )
            .bind(chain_id)
            .bind(finality.wallet_address)
            .fetch_optional(&mut *transaction)
            .await
            .map_err(|e| anyhow::anyhow!("Failed to lock finality nonce ledger: {e}"))?
            .ok_or_else(|| anyhow::anyhow!("Canonical nonce ledger is not initialized"))?;
        anyhow::ensure!(
            manifest_version == finality.manifest_version
                && manifest_digest == finality.manifest_digest,
            "Verified finality manifest identity changed"
        );
        anyhow::ensure!(
            stored_nonce == nonce,
            "Finalized nonce {} does not match canonical nonce {stored_nonce}",
            finality.nonce
        );
        let stored_tip = sqlx::query_as::<_, (i64, String, String, i64, Option<String>, String)>(
            "
            SELECT number, hash, parent_hash, timestamp, base_fee_per_gas, manifest_digest
            FROM execution_verified_finalized_header
            WHERE chain_id = $1 AND wallet_address = $2
            ORDER BY number DESC
            LIMIT 1
            ",
        )
        .bind(chain_id)
        .bind(finality.wallet_address)
        .fetch_one(&mut *transaction)
        .await
        .map_err(|e| anyhow::anyhow!("Failed to lock finalized header tip: {e}"))?;

View on GitHub (pinned to 18893faf8b)

Solutions

  1. Ensure finality transitions are recorded strictly in nonce order; replay from the block height corresponding to the stored next_canonical_nonce, not earlier
  2. Check for concurrent writers and serialize recording of finality per (chain_id, wallet_address) with a single process or lock
  3. Inspect the stored next_canonical_nonce versus the transaction's on-chain nonce; if state diverged legitimately, rebuild/reconcile the ledger from on-chain data
  4. Do not re-record already-finalized transactions; deduplicate before calling record_execution_finality_verified

Example fix

// before: replaying old finality out of order
cache.record_execution_finality_verified(&finality_with_nonce_7).await?;

// after: skip transitions already covered by the canonical nonce
if finality.nonce >= ledger.next_canonical_nonce {
    cache.record_execution_finality_verified(&finality).await?;
}
Defensive patterns

Strategy: validation

Validate before calling

let stored: i64 = sqlx::query_scalar(
    "SELECT next_canonical_nonce FROM execution_verification_nonce WHERE chain_id = $1 AND wallet_address = $2"
)
.bind(chain_id).bind(wallet_address).fetch_one(&pool).await?;
anyhow::ensure!(
    stored == finality.nonce,
    "Skip or reconcile: finality nonce {} != canonical {}",
    finality.nonce, stored
);

Try / catch

match cache.record_execution_finality_verified(&finality).await {
    Err(e) if e.to_string().contains("does not match canonical nonce") => {
        // resync the recorder to the stored canonical nonce before retrying
    }
    result => result?,
}

Prevention

When it happens

Trigger: Calling record_execution_finality_verified with a finality whose finality.nonce is lower (already-recorded/replayed finality) or higher (skipped transactions, nonce gap) than the stored canonical nonce for that (chain_id, wallet_address). Also occurs when finality transitions are recorded out of order or a prior nonce consumption was rolled back while the ledger advanced.

Common situations: Replaying historical blocks/finality events into an already up-to-date database; a transaction being replaced or dropped causing a nonce gap; running multiple recorder processes concurrently so nonces are finalized out of order; restoring a database from a backup taken at an earlier nonce.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/7890bc1e155c482c. Report an issue: GitHub.