nautechsystems/nautilus_trader · error · anyhow::Error

Failed to validate finalized header ledger: {e}

Error message

Failed to validate finalized header ledger: {e}

What it means

This error wraps a sqlx/PostgreSQL failure on the read-back verification SELECT that follows each header insert during ledger extension. After inserting a header, the code immediately re-selects the stored (hash, parent_hash, timestamp, base_fee_per_gas, manifest_digest) row by (chain_id, wallet_address, number) using `fetch_one` and maps any query error to this message. The read-back exists to prove the durable row matches what was written; failing to even execute the read (connection, timeout, missing table) surfaces here.

Source

Thrown at crates/adapters/blockchain/src/cache/database.rs:4497

            .bind(timestamp)
            .bind(&base_fee)
            .bind(bootstrap.manifest_digest)
            .execute(&mut *transaction)
            .await
            .map_err(|e| anyhow::anyhow!("Failed to extend finalized header ledger: {e}"))?;
            let stored = sqlx::query_as::<_, (String, String, i64, Option<String>, String)>(
                "
                SELECT hash, parent_hash, timestamp, base_fee_per_gas, manifest_digest
                FROM execution_verified_finalized_header
                WHERE chain_id = $1 AND wallet_address = $2 AND number = $3
                ",
            )
            .bind(chain_id)
            .bind(bootstrap.wallet_address)
            .bind(number)
            .fetch_one(&mut *transaction)
            .await
            .map_err(|e| anyhow::anyhow!("Failed to validate finalized header ledger: {e}"))?;
            anyhow::ensure!(
                stored
                    == (
                        header.hash.clone(),
                        header.parent_hash.clone(),
                        timestamp,
                        base_fee,
                        bootstrap.manifest_digest.to_string(),
                    ),
                "Finalized header ledger conflicts at height {}",
                header.number
            );
        }

        let finalized_height = bootstrap
            .finalized_headers
            .last()
            .expect("verified finalized headers are nonempty")

View on GitHub (pinned to 18893faf8b)

Solutions

  1. Inspect the wrapped `{e}` cause to determine whether it is connectivity, timeout, permission, or missing relation.
  2. Run pending schema migrations so `execution_verified_finalized_header` exists with the expected columns.
  3. Grant SELECT on the table to the database role used by the adapter, and verify you are not pointed at a read-only replica during a write transaction.
  4. Increase statement/idle-in-transaction timeouts or shrink the batch so the insert+verify pair completes within limits.
  5. Retry the bootstrap on transient connection errors; the transaction design makes it safe to re-run.

Example fix

// before: verification read with no timeout budget on huge batches
let stored = sqlx::query_as::<_, Row>(SELECT ...).fetch_one(&mut *tx).await?;

// after: give the verification read an explicit statement timeout
sqlx::query("SET LOCAL statement_timeout = '30s'").execute(&mut *tx).await?;
let stored = sqlx::query_as::<_, Row>(SELECT ...)
    .fetch_one(&mut *tx).await
    .context("ledger verification read failed")?;
Defensive patterns

Strategy: try-catch

Validate before calling

let can_read = sqlx::query(
    "SELECT has_table_privilege(current_user, 'execution_verified_finalized_header', 'SELECT')",
).fetch_one(&pool).await?;
if !can_read.get::<bool, _>(0) {
    return Err(anyhow!("db role lacks SELECT on execution_verified_finalized_header"));
}

Try / catch

match verify_stored_header(&mut tx, chain_id, wallet, number).await {
    Err(e) if e.to_string().contains("Failed to validate finalized header ledger") => {
        // check connectivity/schema/permissions, then retry whole bootstrap
        Err(anyhow!("verification read failed: {e:#}")).context("bootstrap aborted")
    }
    other => other,
}

Prevention

When it happens

Trigger: PostgreSQL errors on the verification SELECT inside the bootstrap transaction: connection loss mid-transaction, statement timeout, relation `execution_verified_finalized_header` missing (schema not migrated), permission denied on SELECT, or the transaction aborted by a prior error leaving the connection in a failed state.

Common situations: Deploying a new adapter version against a database that has not run migrations; read-only replica or restricted role lacking SELECT on the table; long-running transaction exceeding idle-in-transaction/statement timeouts during a big bootstrap; transient network failure between insert and verification.

Understand the failure class

Background: Database query failed: Internal Server Error 500s wrapping SQL, Prisma, and connection failures — what to check first — this error's family across 16 libraries.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@18893faf8b (2026-09-08). Data as JSON: /api/errors/fa3d0f9b765c25c0. Report an issue: GitHub.