linera-io/linera-protocol · critical

EVM scan loop exited unexpectedly: {result:?}

Error message

EVM scan loop exited unexpectedly: {result:?}

What it means

In serve_loop's tokio::select!, the EVM scan loop task exited. This loop periodically scans the EVM chain for deposit events (eth_getLogs within max_log_block_range windows); its exit — clean return or panic — fails the whole relay loop by design, because deposits would otherwise go unprocessed silently.

Source

Thrown at linera-bridge/src/relay/mod.rs:508

    );

    // Safety-net inbox drain. Notification delivery can be missed (relay down,
    // stream hiccup), and a missed `NewIncomingBundle` would otherwise strand
    // its messages until the next one. Reuse the monitor's scan interval as the
    // drain cadence; an empty inbox makes `process_inbox` a cheap no-op.
    let mut inbox_drain_interval = tokio::time::interval(monitor_scan_interval);
    // Consume the immediate first tick — the startup drain above already ran.
    inbox_drain_interval.tick().await;

    // ── Main loop: process chain operations + notifications ──
    tracing::info!("Listening for chain operations and notifications...");
    loop {
        tokio::select! {
            result = &mut chain_listener_handle => {
                anyhow::bail!("Chain listener exited unexpectedly: {result:?}");
            }
            result = &mut evm_scan_handle => {
                anyhow::bail!("EVM scan loop exited unexpectedly: {result:?}");
            }
            result = &mut linera_scan_handle => {
                anyhow::bail!("Linera scan loop exited unexpectedly: {result:?}");
            }
            result = &mut retry_handle => {
                anyhow::bail!("Retry loop exited unexpectedly: {result:?}");
            }
            result = &mut http_server_handle => {
                anyhow::bail!("HTTP server exited unexpectedly: {result:?}");
            }
            result = &mut admin_server_handle => {
                anyhow::bail!("Admin HTTP server exited unexpectedly: {result:?}");
            }
            _ = inbox_drain_interval.tick() => {
                // Periodic safety net for a missed `NewIncomingBundle`: sync and
                // drain the inbox so stranded messages (e.g. user burns) are
                // eventually processed even without a fresh notification.
                if let Err(e) = chain_client.synchronize_from_validators().await {

View on GitHub (pinned to 6c226ddcb3)

Solutions

1. Check relay logs for the scan loop's own error before the bail — it logs the underlying RPC/decoding failure.
2. If RPC rate-limiting: switch to a better provider or raise rate limits; verify max_log_block_range is within the provider's allowed eth_getLogs window.
3. If a log decoding panic: capture the offending log from the backtrace context and fix the decoder/ABI, then restart the relay.
4. Restart the relay process; scanning resumes from persisted heights without duplicate processing.

Example fix

// before
result = &mut evm_scan_handle => {
    anyhow::bail!("EVM scan loop exited unexpectedly: {result:?}");
}

// after: keep the raw error chain visible
result = &mut evm_scan_handle => {
    anyhow::bail!("EVM scan loop exited unexpectedly: {result:?}; \
                 check RPC availability and eth_getLogs range limits");
}

// Before long runs: verify the RPC honors your scan window once at startup:
let bn = provider.get_block_number().await?;
let _ = provider.get_logs(&Filter::new().from_block(bn - max_log_block_range).to_block(bn)).await?; // fails fast if range is rejected

// The scan loop's exit is fatal to serve_loop; catch at process level and restart.
// Distinguish cause for operators:
match result { Err(j) if j.is_panic() => log panic backtrace, _ => log last RPC error }

When it happens

Trigger: The EVM scan future returns Err (persistent eth_getLogs failure after its internal retries, e.g. RPC endpoint down or rate-limiting for the whole scan interval) or panics on a code path like log decoding; the scan task also gets woken by scan_notify for burn events, so bugs in that path can kill it.

Common situations: Public RPC provider outage or sustained 429 rate limiting; an EVM log that fails decoding after a contract upgrade (ABI drift); misconfigured max_log_block_range exceeding the provider's cap causing repeated RPC errors.

Related errors

• Chain listener exited unexpectedly: {result:?}
• Linera scan loop exited unexpectedly: {result:?}
• Retry loop exited unexpectedly: {result:?}
• HTTP server exited unexpectedly: {result:?}
• Admin HTTP server exited unexpectedly: {result:?}

AI-assisted analysis of linera-io/linera-protocol@6c226ddcb3 (2026-08-22). Data as JSON: /api/errors/7906e25ed0b826a4. Report an issue: GitHub.