{"record":{"id":"6f128a0caefcf50e","repo":"nautechsystems/nautilus_trader","slug":"failed-to-start-recoverable-execution-transition","errorCode":null,"errorMessage":"Failed to start recoverable execution transition: {e}","messagePattern":"Failed to start recoverable execution transition: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/adapters/blockchain/src/cache/database.rs","lineNumber":6512,"sourceCode":"                .context(\"failed to record the verified replacement transition\")?;\n            }\n        }\n\n        transaction\n            .commit()\n            .await\n            .context(\"failed to commit verified replacement scan\")?;\n        Ok(())\n    }\n\n    /// Releases an intent when no broadcast attempt can have occurred.\n    ///\n    /// # Errors\n    ///\n    /// Returns an error if the intent advanced to broadcast or persistence fails.\n    pub async fn mark_execution_intent_recoverable(&self, intent_id: i64) -> anyhow::Result<()> {\n        let mut transaction = self.pool.begin().await.map_err(|e| {\n            anyhow::anyhow!(\"Failed to start recoverable execution transition: {e}\")\n        })?;\n        let current_status = sqlx::query_scalar::<_, String>(\n            \"SELECT status FROM execution_intent WHERE id = $1 FOR UPDATE\",\n        )\n        .bind(intent_id)\n        .fetch_optional(&mut *transaction)\n        .await\n        .map_err(|e| anyhow::anyhow!(\"Failed to lock recoverable execution intent: {e}\"))?\n        .ok_or_else(|| anyhow::anyhow!(\"Execution intent {intent_id} was not found\"))?;\n        anyhow::ensure!(\n            current_status == \"prepared\",\n            \"Execution intent {intent_id} is {current_status}, not recoverable before signing\"\n        );\n        let result = sqlx::query(\n            \"\n            UPDATE execution_intent\n            SET status = 'recoverable', active = FALSE, updated_at = NOW()\n            WHERE id = $1 AND status = 'prepared'","sourceCodeStart":6494,"sourceCodeEnd":6530,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/blockchain/src/cache/database.rs#L6494-L6530","documentation":"`mark_execution_intent_recoverable` begins by opening a PostgreSQL transaction via `self.pool.begin()`. If acquiring a connection/starting the transaction fails (pool exhausted, database down, TLS/network error), the underlying sqlx error is wrapped into this anyhow error and returned before any intent state is touched.","triggerScenarios":"Any call to `mark_execution_intent_recoverable(intent_id)` while the database connection pool is exhausted, the database is unreachable, credentials are invalid, or the server rejects `BEGIN`.","commonSituations":"Connection pool saturation under load; PostgreSQL restart or failover; network partition between the adapter and the database; wrong DSN or expired credentials in the environment.","solutions":["Check database connectivity and credentials, then retry the operation — it is safe to retry since nothing was modified.","Increase the sqlx pool size or add acquire-timeout headroom to handle pool exhaustion.","Inspect the wrapped inner error (`{e}`) to distinguish network vs auth vs server errors.","Add connection health checks / keepalives so stale connections are replaced before `begin()`."],"exampleFix":"// before: no retry around transient pool errors\nfor id in intents { db.mark_execution_intent_recoverable(id).await?; }\n// after: bounded retry on transient failures\nfor id in intents {\n    for _ in 0..3 {\n        match db.mark_execution_intent_recoverable(id).await {\n            Ok(()) => break,\n            Err(e) if is_transient_db_error(&e) => tokio::time::sleep(Duration::from_millis(200)).await,\n            Err(e) => return Err(e),\n        }\n    }\n}","handlingStrategy":"retry","validationCode":"sqlx::query(\"SELECT 1\").execute(&pool).await.map_err(|e| anyhow!(\"database unreachable: {e}\"))?;","typeGuard":null,"tryCatchPattern":"match db.mark_execution_intent_recoverable(id).await {\n    Err(e) if is_transient_db_error(&e) => { backoff_and_retry(id, 3).await?; }\n    Err(e) => return Err(e),\n    Ok(()) => {},\n}","preventionTips":["Size the connection pool for peak concurrency and set a sane acquire timeout.","Monitor pool wait metrics to catch saturation before it surfaces as begin() failures.","Validate DSN/credentials at startup so auth failures do not appear mid-lifecycle.","Make recoverable-transition calls idempotent so they are safe to retry after transient DB errors."],"tags":["database","connection","transient"],"backgroundTag":"database-query-failed","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}