{"record":{"id":"c1ad970ae4df07ac","repo":"nautechsystems/nautilus_trader","slug":"failed-to-start-verified-finality-transition-e","errorCode":null,"errorMessage":"Failed to start verified finality transition: {e}","messagePattern":"Failed to start verified finality transition: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/adapters/blockchain/src/cache/database.rs","lineNumber":6931,"sourceCode":"                    .finalized_headers\n                    .last()\n                    .is_some_and(|header| header.number >= finality.block_number),\n            \"Verified finality headers must form a continuous chain through the inclusion height\"\n        );\n        let chain_id = i32::try_from(finality.chain_id)\n            .context(\"Verification chain ID exceeds PostgreSQL INTEGER\")?;\n        let nonce =\n            i64::try_from(finality.nonce).context(\"Execution nonce exceeds PostgreSQL BIGINT\")?;\n        let next_nonce = nonce\n            .checked_add(1)\n            .ok_or_else(|| anyhow::anyhow!(\"Canonical nonce overflow\"))?;\n        let block_number = i64::try_from(finality.block_number)\n            .context(\"Finality block exceeds PostgreSQL BIGINT\")?;\n        let gas_used = i64::try_from(finality.gas_used)\n            .context(\"Finality gas used exceeds PostgreSQL BIGINT\")?;\n        let mut transaction =\n            self.pool.begin().await.map_err(|e| {\n                anyhow::anyhow!(\"Failed to start verified finality transition: {e}\")\n            })?;\n        let (manifest_version, manifest_digest, stored_nonce, revision) =\n            sqlx::query_as::<_, (String, String, i64, i64)>(\n                \"\n                SELECT manifest_version, manifest_digest, next_canonical_nonce, revision\n                FROM execution_verification_nonce\n                WHERE chain_id = $1 AND wallet_address = $2\n                FOR UPDATE\n                \",\n            )\n            .bind(chain_id)\n            .bind(finality.wallet_address)\n            .fetch_optional(&mut *transaction)\n            .await\n            .map_err(|e| anyhow::anyhow!(\"Failed to lock finality nonce ledger: {e}\"))?\n            .ok_or_else(|| anyhow::anyhow!(\"Canonical nonce ledger is not initialized\"))?;\n        anyhow::ensure!(\n            manifest_version == finality.manifest_version","sourceCodeStart":6913,"sourceCodeEnd":6949,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/blockchain/src/cache/database.rs#L6913-L6949","documentation":"Thrown when `self.pool.begin()` fails, i.e. the database pool could not open a new SQL transaction to record a verified finality transition. The original sqlx error is wrapped with this context message so callers can tell which stage of finality persistence failed. No ledger rows are modified because the transaction never started.","triggerScenarios":"Calling record_execution_finality_verified (directly or via the execution pipeline) while the PostgreSQL pool is unavailable: pool exhausted (all connections checked out), database down/restarting, network partition to the DB, TLS/auth failure at connection time, or pool acquisition timeout.","commonSituations":"Under heavy load the pool's max_connections is exhausted by concurrent finality recordings; a failover or maintenance window dropped DB connectivity; connection lifetime settings closed idle connections while traffic resumed; misconfigured DATABASE_URL credentials after a rotation.","solutions":["Check database connectivity and pool health (DB reachable, credentials valid, listener up)","Increase pool max_connections or acquire_timeout in sqlx pool config if the pool is exhausted under load","Retry the finality recording with backoff — it is transactional and safe to re-attempt once connectivity is restored","Inspect the wrapped sqlx error (`{e}` in the message) for the root cause (auth, TLS, timeout, connection refused)"],"exampleFix":"// before\n// pool configured with defaults, starves under concurrent finality writes\nPgPoolOptions::new().max_connections(5).connect(url).await\n// after\nPgPoolOptions::new()\n    .max_connections(20)\n    .acquire_timeout(Duration::from_secs(10))\n    .connect(url)\n    .await","handlingStrategy":"retry","validationCode":"// Rust: probe pool health before high-stakes finality writes\nlet healthy = sqlx::query(\"SELECT 1\").execute(&db.pool).await.is_ok();\nensure!(healthy, \"database pool unavailable; defer finality recording\");","typeGuard":null,"tryCatchPattern":"// Retry transactional start with bounded backoff\nfor attempt in 0..3 {\n    match db.record_execution_finality_verified(&finality).await {\n        Err(e) if e.to_string().contains(\"Failed to start verified finality transition\")\n            && attempt < 2 => tokio::time::sleep(backoff(attempt)).await,\n        other => { other?; break }\n    }\n}","preventionTips":["Size the pool (max_connections) above peak concurrent finality writers and set a sane acquire_timeout","Monitor DB connectivity and fail over before issuing finality writes","Because the operation is transactional, safe retries are fine — retry idempotently with backoff","Alert on the wrapped sqlx root cause (auth/TLS/refused) to distinguish config drift from transient outages"],"tags":["database","transaction","postgresql","network"],"backgroundTag":"database-query-failed","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}