What causes "Failed to validate nonce recovery ownership: {e}" in nautechsystems/nautilus_trader?

Inspect the wrapped sqlx error in the {e} message to identify the root cause (connection, timeout, permission, or schema).

nautechsystems/nautilus_trader · error · anyhow::Error

Failed to validate nonce recovery ownership: {e}

Error message

Failed to validate nonce recovery ownership: {e}

What it means

The ownership-validation query that checks for an active execution intent owning the recovered nonce failed at the database level; crates/adapters/blockchain/src/cache/database.rs:4072 wraps the sqlx error with this message. This is an infrastructure/database failure (connectivity, permissions, lock contention, schema mismatch) rather than a business-rule violation — the query itself could not complete.

Source

Thrown at crates/adapters/blockchain/src/cache/database.rs:4072

                                   + (hash.sealed_transaction IS NOT NULL)::INTEGER) = 1
                              AND hash.status IN (
                                  'broadcast', 'included', 'replaced', 'dropped', 'reorged'
                              )
                        )
                    FROM execution_intent AS intent
                    LEFT JOIN execution_transaction_hash AS hash
                        ON hash.intent_id = intent.id
                    WHERE intent.chain_id = $1
                      AND intent.wallet_address = $2
                      AND intent.active
                    GROUP BY intent.id, intent.nonce, intent.status
                    ",
                )
                .bind(chain_id)
                .bind(bootstrap.wallet_address)
                .fetch_optional(&mut *transaction)
                .await
                .map_err(|e| anyhow::anyhow!("Failed to validate nonce recovery ownership: {e}"))?;
                let Some((intent_nonce, intent_status, payload_count)) = recovery else {
                    anyhow::bail!(
                        "Verified finalized transaction count advanced without an active owned intent"
                    );
                };
                anyhow::ensure!(
                    intent_nonce == Some(stored_nonce)
                        && matches!(
                            intent_status.as_str(),
                            "broadcast" | "included" | "replaced" | "dropped" | "reorged"
                        )
                        && payload_count == 1,
                    "Verified finalized transaction count advanced without one recoverable retained payload at the durable nonce"
                );
            }
            revision
        } else {
            anyhow::ensure!(

View on GitHub (pinned to 18893faf8b)

Solutions

Inspect the wrapped sqlx error in the {e} message to identify the root cause (connection, timeout, permission, or schema).
Test connectivity and retry after transient failures (connection reset, server restart).
Verify the schema matches the expected verification schema version (execution_schema_version for 'evm_execution_verification') and run pending migrations.
Check for lock contention on execution_intent/execution_transaction_hash and long-running transactions blocking the ACCESS EXCLUSIVE lock.
Confirm the database role has SELECT on execution_intent and execution_transaction_hash.

Example fix

// before
// bootstrap retried immediately on any db failure, masking the root sqlx error
run_bootstrap().await?;

// after
match run_bootstrap().await {
    Err(e) if e.to_string().contains("Failed to validate nonce recovery ownership") => {
        // log the inner sqlx error, check connectivity/schema, retry with backoff
        tracing::error!(error = ?e, "nonce recovery ownership check failed");
        retry_with_backoff(run_bootstrap).await?
    }
    other => other?,
}

Defensive patterns

Strategy: retry

Validate before calling

// pre-check the tables are queryable before entering bootstrap
sqlx::query("SELECT 1 FROM execution_intent LIMIT 1").execute(&pool).await?;
sqlx::query("SELECT 1 FROM execution_transaction_hash LIMIT 1").execute(&pool).await?;

Try / catch

match bootstrap_verification(...).await {
    Err(e) if e.to_string().contains("Failed to validate nonce recovery ownership") => {
        // inspect inner sqlx error; retry with backoff for transient errors, fail fast on schema/permission errors
        if is_transient(&e) { retry_with_backoff(|| bootstrap_verification(...), 3).await? } else { return Err(e) }
    }
    other => other?,
}

Prevention

Run schema version checks and migrations before bootstrap
Use connection-pool health checks and sensible statement timeouts
Grant the database role SELECT on execution_intent and execution_transaction_hash
Monitor and minimize long-running transactions that block the verification table locks

When it happens

Trigger: The SELECT over execution_intent joined with execution_transaction_hash returns a database error during the nonce recovery branch of verification bootstrap: connection drop, statement timeout under the ACCESS EXCLUSIVE table lock, missing tables/columns (schema version mismatch), or insufficient privileges.

Common situations: Transient Postgres restart or failover mid-bootstrap; another migration dropped or renamed execution_intent columns; lock queue behind a long-running transaction holding the verification tables; connection pool exhausted; user lacking SELECT on the tables.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/3f36cd32169496de. Report an issue: GitHub.