nautechsystems/nautilus_trader · error · anyhow::Error

Canonical head changed during signer-nonce replacement scan

Error message

Canonical head changed during signer-nonce replacement scan

What it means

find_nonce_transaction scans blocks from a start height to the head looking for the signer's nonce, then re-fetches the head block to confirm the scan covered a stable chain view. If the re-fetched head hash differs from the hash captured before scanning, blocks moved (a reorg or rapid head advance) during the scan, so any result would be unreliable and the scan aborts.

Source

Thrown at crates/adapters/blockchain/src/execution/client.rs:1813

        from_block: u64,
    ) -> anyhow::Result<Option<RpcTransaction>> {
        let head = self.http_rpc_client.latest_block().await?;
        let mut found = None;

        for number in from_block..=head.number {
            let block = self.http_rpc_client.block_by_number(number, true).await?;
            if let Some(transaction) = block.transactions.into_iter().find(|transaction| {
                transaction.from == self.wallet_address && transaction.nonce == nonce
            }) {
                found = Some(transaction);
                break;
            }
        }
        let stable_head = self
            .http_rpc_client
            .block_by_number(head.number, false)
            .await?;
        anyhow::ensure!(
            stable_head.hash == head.hash,
            "Canonical head changed during signer-nonce replacement scan"
        );
        Ok(found)
    }

    fn release_slot(&self) {
        *self.in_flight.lock().expect("in-flight mutex poisoned") = None;
    }
}

fn current_execution_hash(
    intent_id: i64,
    hashes: &[ExecutionTransactionHashRow],
) -> anyhow::Result<&ExecutionTransactionHashRow> {
    let mut current = hashes.iter().filter(|row| row.current);
    let row = current
        .next()

View on GitHub (pinned to 2114cf6f76)

Solutions

  1. Retry the operation: the scan restarts against the new head and usually succeeds once the view is stable
  2. Narrow the scan range (start closer to the current head) so it completes within one or two blocks
  3. Use an RPC endpoint with sticky sessions/consistent heads if the error repeats constantly
  4. If it recurs on every attempt, check the network for an ongoing reorg incident

Example fix

# before: single scan attempt
result = find_nonce_transaction(nonce, from_block)  # raises on head change

# after: retry until the chain view stays stable
for _ in range(5):
    try:
        result = find_nonce_transaction(nonce, from_block)
        break
    except Exception as e:
        if 'Canonical head changed' not in str(e):
            raise
        from_block = max(from_block, latest_height() - 64)
Defensive patterns

Strategy: retry

Validate before calling

# Bound the scan window to a couple of recent blocks to make head moves unlikely
latest = w3.eth.get_block('latest')
from_block = max(from_block, latest.number - 12)  # ~2-3s of blocks on L2
assert w3.eth.get_block(latest.number)['hash'] == latest.hash, 'unstable node view'

Try / catch

for attempt in range(5):
    try:
        found = find_nonce_transaction(nonce, from_block)
        break
    except Exception as e:
        if 'Canonical head changed' not in str(e):
            raise
        time.sleep(2)  # head moved mid-scan; retry against the new head
else:
    raise RuntimeError('nonce scan could not get a stable head')

Prevention

When it happens

Trigger: Replacement/recovery scans (find_nonce_transaction) executed while the chain head advances or reorganizes: common on busy networks where several blocks are produced during a multi-block scan, or during a live reorg event.

Common situations: Scanning long block ranges after downtime on fast L2s (block times under a second); RPC nodes behind load balancers serving inconsistent heads; genuine reorg storms.

Related errors

AI-assisted analysis of nautechsystems/nautilus_trader@2114cf6f76 (2026-08-21). Data as JSON: /api/errors/64dd820a0ba6442e. Report an issue: GitHub.