{"record":{"id":"1666a68ff7f36034","repo":"nautechsystems/nautilus_trader","slug":"failed-to-start-verified-nonce-assignment-e","errorCode":null,"errorMessage":"Failed to start verified nonce assignment: {e}","messagePattern":"Failed to start verified nonce assignment: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/adapters/blockchain/src/cache/database.rs","lineNumber":5912,"sourceCode":"            \"Verified nonce assignment requires decision evidence\"\n        );\n        anyhow::ensure!(\n            assignment.provider_ids.len() == 3 && assignment.operator_ids.len() == 3,\n            \"Verified nonce assignment requires exactly three provider and operator IDs\"\n        );\n        anyhow::ensure!(\n            assignment.failure_domain_ids.len() >= 3,\n            \"Verified nonce assignment requires the configured failure domains\"\n        );\n        let chain_id = i32::try_from(assignment.chain_id)\n            .context(\"Verification chain ID exceeds PostgreSQL INTEGER\")?;\n        let nonce =\n            i64::try_from(assignment.nonce).context(\"Execution nonce exceeds PostgreSQL BIGINT\")?;\n        let mut transaction = self\n            .pool\n            .begin()\n            .await\n            .map_err(|e| anyhow::anyhow!(\"Failed to start verified nonce assignment: {e}\"))?;\n        let (manifest_version, manifest_digest, next_nonce, revision) =\n            sqlx::query_as::<_, (String, String, i64, i64)>(\n                \"\n                SELECT manifest_version, manifest_digest, next_canonical_nonce, revision\n                FROM execution_verification_nonce\n                WHERE chain_id = $1 AND wallet_address = $2\n                FOR UPDATE\n                \",\n            )\n            .bind(chain_id)\n            .bind(assignment.wallet_address)\n            .fetch_optional(&mut *transaction)\n            .await\n            .map_err(|e| anyhow::anyhow!(\"Failed to lock canonical nonce ledger: {e}\"))?\n            .ok_or_else(|| anyhow::anyhow!(\"Canonical nonce ledger is not initialized\"))?;\n        anyhow::ensure!(\n            manifest_version == assignment.manifest_version\n                && manifest_digest == assignment.manifest_digest,","sourceCodeStart":5894,"sourceCodeEnd":5930,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/blockchain/src/cache/database.rs#L5894-L5930","documentation":"This error is raised while beginning a PostgreSQL transaction (via sqlx `pool.begin()`) to record a verified execution nonce assignment. It wraps the underlying connection/acquisition error with context so callers know the failure happened at the very start of the nonce-assignment flow. If the pool cannot establish or acquire a connection, the assignment is aborted before any row is read or written.","triggerScenarios":"Calling the verified-nonce-assignment persistence path when `self.pool.begin()` fails: database unreachable, connection pool exhausted (all checkouts in use / timeout), TLS or auth failure, or the database was restarted between pool creation and this call.","commonSituations":"Postgres container down or restarted during a trading session; pool max_connections too low for concurrent strategies; network partition to the DB host; wrong DSN credentials in environment config; firewall/idle timeouts killing pooled connections.","solutions":["Check the wrapped source error (`{e}` in the message / the anyhow chain) for the root cause: connection refused, auth failed, or pool timeout.","Verify the PostgreSQL host is reachable and healthy (`pg_isready`, docker ps, cloud console).","Increase pool limits or checkout timeout in the sqlx PoolOptions configuration to match concurrency.","If pool timeout, look for connection leaks (transactions not committed/rolled back elsewhere).","Confirm DSN/env vars (host, port, user, password, sslmode) are correct for the environment."],"exampleFix":"// before\n DatabaseConfig::new(dsn).max_connections(5)\n// after\n DatabaseConfig::new(dsn).max_connections(20).acquire_timeout(Duration::from_secs(10))","handlingStrategy":"try-catch","validationCode":"let healthy = sqlx::query(\"SELECT 1\").fetch_one(&pool).await.is_ok();\nif !healthy { return Err(anyhow!(\"database pool unavailable before nonce assignment\")); }","typeGuard":null,"tryCatchPattern":"match db.assign_verified_nonce(&assignment).await {\n    Ok(()) => info!(\"nonce assigned\"),\n    Err(e) if e.to_string().contains(\"Failed to start verified nonce assignment\") => {\n        warn!(\"DB transaction could not start; retrying after health check\");\n        wait_for_db(&pool).await?;\n        retry(assignment)\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Run a startup health check (SELECT 1) before processing work that needs the database.","Size the pool (max_connections, acquire_timeout) for your actual concurrency.","Monitor pool checkout saturation and DB connectivity with metrics/alerts.","Use bounded retry with backoff around DB-bound operations.","Use TCP keepalives on Postgres connections to survive idle network timeouts."],"tags":["database","postgres","connection-pool","transaction"],"backgroundTag":"database-query-failed","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}