{"record":{"id":"1bba3767d07831f6","repo":"nautechsystems/nautilus_trader","slug":"failed-to-lock-finality-nonce-ledger-e","errorCode":null,"errorMessage":"Failed to lock finality nonce ledger: {e}","messagePattern":"Failed to lock finality nonce ledger: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/adapters/blockchain/src/cache/database.rs","lineNumber":6946,"sourceCode":"            .context(\"Finality gas used exceeds PostgreSQL BIGINT\")?;\n        let mut transaction =\n            self.pool.begin().await.map_err(|e| {\n                anyhow::anyhow!(\"Failed to start verified finality transition: {e}\")\n            })?;\n        let (manifest_version, manifest_digest, stored_nonce, revision) =\n            sqlx::query_as::<_, (String, String, i64, i64)>(\n                \"\n                SELECT manifest_version, manifest_digest, next_canonical_nonce, revision\n                FROM execution_verification_nonce\n                WHERE chain_id = $1 AND wallet_address = $2\n                FOR UPDATE\n                \",\n            )\n            .bind(chain_id)\n            .bind(finality.wallet_address)\n            .fetch_optional(&mut *transaction)\n            .await\n            .map_err(|e| anyhow::anyhow!(\"Failed to lock finality nonce ledger: {e}\"))?\n            .ok_or_else(|| anyhow::anyhow!(\"Canonical nonce ledger is not initialized\"))?;\n        anyhow::ensure!(\n            manifest_version == finality.manifest_version\n                && manifest_digest == finality.manifest_digest,\n            \"Verified finality manifest identity changed\"\n        );\n        anyhow::ensure!(\n            stored_nonce == nonce,\n            \"Finalized nonce {} does not match canonical nonce {stored_nonce}\",\n            finality.nonce\n        );\n        let stored_tip = sqlx::query_as::<_, (i64, String, String, i64, Option<String>, String)>(\n            \"\n            SELECT number, hash, parent_hash, timestamp, base_fee_per_gas, manifest_digest\n            FROM execution_verified_finalized_header\n            WHERE chain_id = $1 AND wallet_address = $2\n            ORDER BY number DESC\n            LIMIT 1","sourceCodeStart":6928,"sourceCodeEnd":6964,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/blockchain/src/cache/database.rs#L6928-L6964","documentation":"Thrown when the `SELECT ... FROM execution_verification_nonce` query inside the verified finality transaction fails at the database level. The sqlx error is wrapped with this context so failures can be attributed to locking/reading the canonical nonce ledger row (selected FOR UPDATE style to serialize nonce advancement). The transaction is rolled back, so no partial state is written.","triggerScenarios":"Calling record_execution_finality_verified when the ledger SELECT fails: connection dropped mid-transaction, statement timeout, deadlock/lock timeout on the ledger row, permission denied on execution_verification_nonce, or malformed query parameters (e.g. wallet binding type error).","commonSituations":"Two concurrent finality recordings for the same wallet contend on the ledger row and hit a lock timeout under load; DB failover killed the in-flight transaction; migrations not applied so execution_verification_nonce does not exist; role lacks SELECT grants after a permissions change.","solutions":["Inspect the wrapped sqlx error (`{e}`) to identify the root cause (lock timeout vs missing table vs permissions vs connection loss)","Retry the finality recording with backoff if it was a transient lock/deadlock conflict","Verify the execution_verification_nonce table exists and migrations ran (missing table would surface here as a query error)","Check database grants for the application role on execution_verification_nonce","Reduce contention by serializing finality recordings per wallet upstream"],"exampleFix":"// before\n// concurrent transitions for same wallet hit lock timeout\nlet _ = tokio::join!(record(&db, f1), record(&db, f2));\n// after\n// serialize per wallet and retry on transient failure\nlet res = retry_backoff(3, || record(&db, f1)).await;","handlingStrategy":"retry","validationCode":"// Rust: verify table exists and grants before recording finality\nsqlx::query(\n    \"SELECT 1 FROM information_schema.tables WHERE table_name = 'execution_verification_nonce'\",\n).fetch_one(&db.pool).await\n    .context(\"execution_verification_nonce table missing; run migrations\")?;","typeGuard":null,"tryCatchPattern":"// Distinguish transient lock errors (retry) from permanent ones (fail fast)\nmatch db.record_execution_finality_verified(&finality).await {\n    Err(e) if is_transient_db_error(&e) => retry_with_backoff(3, || record(&db, &finality)).await?,\n    Err(e) if e.to_string().contains(\"does not exist\") => run_migrations()?,\n    other => other?,\n}","preventionTips":["Apply migrations (including execution_verification_nonce) as part of deployment health checks","Grant the app role SELECT/UPDATE on execution_verification_nonce and verify after role changes","Serialize finality recordings per wallet to avoid lock contention on the ledger row","Set explicit lock_timeout/statement_timeout and treat those as retryable transient errors"],"tags":["database","postgresql","locking","transaction"],"backgroundTag":"database-query-failed","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}