linera-io/linera-protocol · error · anyhow

Failed to start {nickname}

Error message

Failed to start {nickname}

What it means

`ensure_grpc_server_has_started` spawns nothing itself but polls the gRPC health service (tonic `HealthClient`) at the given port up to 10 times with growing sleeps (100ms + i*500ms), logging 'Waiting for {nickname} to start' each round. If the service never reports `Serving`, it bails with this message — the preceding warn logs are the only diagnostics, so the actual crash reason lives in the spawned process.

Source

Thrown at linera-service/src/cli_wrappers/local_net.rs:925

                    .context("endpoint should always parse")?
                    .tls_config(tls_config)?
            }
            _ => bail!("Only supported scheme are http and https"),
        };
        let connection = endpoint.connect_lazy();
        let mut client = HealthClient::new(connection);
        linera_base::time::timer::sleep(Duration::from_millis(100)).await;
        for i in 0..10 {
            linera_base::time::timer::sleep(Duration::from_millis(i * 500)).await;
            let result = client.check(HealthCheckRequest::default()).await;
            if result.is_ok() && result.unwrap().get_ref().status() == ServingStatus::Serving {
                info!(?port, "Successfully started {nickname}");
                return Ok(());
            } else {
                warn!("Waiting for {nickname} to start");
            }
        }
        bail!("Failed to start {nickname}");
    }

    async fn ensure_simple_server_has_started(
        nickname: &str,
        port: usize,
        protocol: &str,
    ) -> Result<()> {
        use linera_core::node::ValidatorNode as _;

        let options = linera_rpc::NodeOptions {
            send_timeout: Duration::from_secs(5),
            recv_timeout: Duration::from_secs(5),
            retry_delay: Duration::from_secs(1),
            max_retries: 1,
            ..Default::default()
        };
        let provider = linera_rpc::simple::SimpleNodeProvider::new(options);
        let address = format!("{protocol}:127.0.0.1:{port}");

View on GitHub (pinned to 6c226ddcb3)

Solutions

  1. Check the warn logs just above the bail — they name the port/nickname that never came up
  2. Free the port (`lsof -i :<port>` then kill the stale process) and retry
  3. Confirm the server binary exists and runs standalone with the same arguments
  4. Tear down and re-create the net: `linera net down` then `linera net up`
Defensive patterns

Strategy: retry

Validate before calling

# Pre-flight: confirm the port is bindable before starting the net
port=9100
if lsof -i :$port >/dev/null 2>&1; then echo "port $port busy"; exit 1; fi

Try / catch

match LocalNet::ensure_grpc_server_has_started(&nickname, port, scheme).await {
    Err(e) if e.to_string().contains("Failed to start") => {
        // helper already retried 10x with backoff; failure is structural:
        // free the port / fix the binary, then rebuild the net
        cleanup_and_recreate_net().await?;
    }
    result => result?,
}

Prevention

When it happens

Trigger: A validator, proxy, or block-exporter gRPC process spawned by the local net never becomes healthy within the ~10-probe budget: it crashed on startup, the port is already taken, the binary is missing, or (on grpcs nets) the TLS setup does not line up.

Common situations: Port already held by a stale process from an earlier aborted run; the server binary missing from PATH/target; slow machines where startup exceeds the fixed polling budget; leftover net not torn down with `linera net down`.

Related errors


AI-assisted analysis of linera-io/linera-protocol@6c226ddcb3 (2026-08-22). Data as JSON: /api/errors/4700c8af5e9f6491. Report an issue: GitHub.