{"record":{"id":"7890bc1e155c482c","repo":"nautechsystems/nautilus_trader","slug":"finalized-nonce-does-not-match-canonical-nonce","errorCode":null,"errorMessage":"Finalized nonce {} does not match canonical nonce {stored_nonce}","messagePattern":"Finalized nonce (.+?) does not match canonical nonce (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/adapters/blockchain/src/cache/database.rs","lineNumber":6953,"sourceCode":"                \"\n                SELECT manifest_version, manifest_digest, next_canonical_nonce, revision\n                FROM execution_verification_nonce\n                WHERE chain_id = $1 AND wallet_address = $2\n                FOR UPDATE\n                \",\n            )\n            .bind(chain_id)\n            .bind(finality.wallet_address)\n            .fetch_optional(&mut *transaction)\n            .await\n            .map_err(|e| anyhow::anyhow!(\"Failed to lock finality nonce ledger: {e}\"))?\n            .ok_or_else(|| anyhow::anyhow!(\"Canonical nonce ledger is not initialized\"))?;\n        anyhow::ensure!(\n            manifest_version == finality.manifest_version\n                && manifest_digest == finality.manifest_digest,\n            \"Verified finality manifest identity changed\"\n        );\n        anyhow::ensure!(\n            stored_nonce == nonce,\n            \"Finalized nonce {} does not match canonical nonce {stored_nonce}\",\n            finality.nonce\n        );\n        let stored_tip = sqlx::query_as::<_, (i64, String, String, i64, Option<String>, String)>(\n            \"\n            SELECT number, hash, parent_hash, timestamp, base_fee_per_gas, manifest_digest\n            FROM execution_verified_finalized_header\n            WHERE chain_id = $1 AND wallet_address = $2\n            ORDER BY number DESC\n            LIMIT 1\n            \",\n        )\n        .bind(chain_id)\n        .bind(finality.wallet_address)\n        .fetch_one(&mut *transaction)\n        .await\n        .map_err(|e| anyhow::anyhow!(\"Failed to lock finalized header tip: {e}\"))?;","sourceCodeStart":6935,"sourceCodeEnd":6971,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/blockchain/src/cache/database.rs#L6935-L6971","documentation":"Thrown by `record_execution_finality_verified` when the nonce of the finalized transaction does not equal the ledger's stored next-canonical nonce (execution_verification_nonce.next_canonical_nonce). The function requires finality.nonce to be consumed exactly in canonical order; any divergence aborts the transaction. This protects against double-spending nonce gaps or recording finality out of order.","triggerScenarios":"Calling record_execution_finality_verified with a finality whose finality.nonce is lower (already-recorded/replayed finality) or higher (skipped transactions, nonce gap) than the stored canonical nonce for that (chain_id, wallet_address). Also occurs when finality transitions are recorded out of order or a prior nonce consumption was rolled back while the ledger advanced.","commonSituations":"Replaying historical blocks/finality events into an already up-to-date database; a transaction being replaced or dropped causing a nonce gap; running multiple recorder processes concurrently so nonces are finalized out of order; restoring a database from a backup taken at an earlier nonce.","solutions":["Ensure finality transitions are recorded strictly in nonce order; replay from the block height corresponding to the stored next_canonical_nonce, not earlier","Check for concurrent writers and serialize recording of finality per (chain_id, wallet_address) with a single process or lock","Inspect the stored next_canonical_nonce versus the transaction's on-chain nonce; if state diverged legitimately, rebuild/reconcile the ledger from on-chain data","Do not re-record already-finalized transactions; deduplicate before calling record_execution_finality_verified"],"exampleFix":"// before: replaying old finality out of order\ncache.record_execution_finality_verified(&finality_with_nonce_7).await?;\n\n// after: skip transitions already covered by the canonical nonce\nif finality.nonce >= ledger.next_canonical_nonce {\n    cache.record_execution_finality_verified(&finality).await?;\n}","handlingStrategy":"validation","validationCode":"let stored: i64 = sqlx::query_scalar(\n    \"SELECT next_canonical_nonce FROM execution_verification_nonce WHERE chain_id = $1 AND wallet_address = $2\"\n)\n.bind(chain_id).bind(wallet_address).fetch_one(&pool).await?;\nanyhow::ensure!(\n    stored == finality.nonce,\n    \"Skip or reconcile: finality nonce {} != canonical {}\",\n    finality.nonce, stored\n);","typeGuard":null,"tryCatchPattern":"match cache.record_execution_finality_verified(&finality).await {\n    Err(e) if e.to_string().contains(\"does not match canonical nonce\") => {\n        // resync the recorder to the stored canonical nonce before retrying\n    }\n    result => result?,\n}","preventionTips":["Record finality transitions strictly in nonce order from a single writer per (chain_id, wallet_address)","On replay, start from the height/nonce corresponding to the stored canonical nonce, not from genesis","Deduplicate already-finalized nonces before submitting them for recording"],"tags":["database","integrity-check","blockchain","rust"],"backgroundTag":"invalid-state-transition","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}