block/buzz · critical · anyhow::Error

Failed to bind health port

Error message

Failed to bind health port {}: {e}

What it means

buzz-relay's serve() binds a dedicated TCP listener on 0.0.0.0:$BUZZ_HEALTH_PORT for health probes before anything else starts. If the kernel refuses the bind (AddrInUse, PermissionDenied, EADDRNOTAVAIL), the map_err wraps the io::Error into anyhow with the port number and the whole process exits at boot. This is a fail-fast startup check: a relay without a reachable health port would be killed by orchestrators anyway.

Solutions

1. Find and stop the occupant: `ss -ltnp 'sport = :<port>'` (or `lsof -i :<port>`), then kill that process or use another port.
2. Set BUZZ_HEALTH_PORT to a free high port in .env (e.g. 3001) and restart the relay.
3. If the port must be <1024, run the relay with CAP_NET_BIND_SERVICE (`setcap cap_net_bind_service=+ep` on the binary) or bind a high port and map it at the load balancer.
4. Make sure BUZZ_HEALTH_PORT differs from the main relay bind port (BUZZ_BIND_ADDR) and the mesh bind port.
5. In containers, ensure only one process binds the health port; remove duplicate port declarations from the pod spec.

Example fix

# before: BUZZ_HEALTH_PORT=3000 already used by a stale relay
ss -ltnp 'sport = :3000'   # shows pid of old process
kill <pid>

# after: or just pick a free port in .env
BUZZ_HEALTH_PORT=3001

# before starting the relay, confirm the health port is free
if ss -ltnH 'sport = :'"$BUZZ_HEALTH_PORT" | grep -q .; then
  echo "health port $BUZZ_HEALTH_PORT already in use" >&2; exit 1
fi

When it happens

Trigger: TcpListener::bind(("0.0.0.0", config.health_port)) at crates/buzz-relay/src/main.rs:1255 fails when: (1) another process (often a second relay instance or a leftover container) already listens on BUZZ_HEALTH_PORT; (2) BUZZ_HEALTH_PORT is set below 1024 and the process lacks CAP_NET_BIND_SERVICE; (3) BUZZ_HEALTH_PORT collides with the relay's own bind_addr port; (4) the host has no IPv4 interface (bind on 0.0.0.0 fails with EADDRNOTAVAIL).

Common situations: Running `just relay` twice in different terminals; a Docker/K8s pod where the health port is also declared in the container spec and a sidecar grabbed it; copying .env from a teammate whose BUZZ_HEALTH_PORT is already taken on the shared dev box; setting BUZZ_HEALTH_PORT=80 in docker-compose without root.

Related errors

• Failed to bind
• mesh endpoint bind on
• BUZZ_UDS_PATH exists but is not a socket
• channel discovery error
• Failed to bind UDS

AI-assisted analysis of block/buzz@ef2aa1ae38 (2026-08-20). Data as JSON: /api/errors/d3ce7333e79e68b6. Report an issue: GitHub.

Appendix: source

Thrown at crates/buzz-relay/src/main.rs:1373

/// `state.rs`) sum to 25s and stay inside the 30s hard drain. Total worst
/// case from SIGTERM to forced exit is 5s + 30s = 35s. Both fit inside the
/// chart's `terminationGracePeriodSeconds: 60` (`deploy/charts/buzz/values.yaml`),
/// which leaves headroom but assumes no `preStop` hook adds further delay.
/// With jitter off (`BUZZ_DRAIN_JITTER_MS=0`, the default) sockets close
/// all-at-once right after the grace, so the per-socket delay collapses to
/// roughly the 5s grace plus the ack wait.
const GRACEFUL_DRAIN_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);

async fn serve(
    router: axum::Router,
    health_router: axum::Router,
    state: Arc<AppState>,
) -> anyhow::Result<()> {
    let config = &state.config;

    let health_listener = tokio::net::TcpListener::bind(("0.0.0.0", config.health_port))
        .await
        .map_err(|e| anyhow::anyhow!("Failed to bind health port {}: {e}", config.health_port))?;
    info!(port = config.health_port, "Health probe listener started");
    tokio::spawn(async move {
        axum::serve(health_listener, health_router).await.ok();
    });

    let (shutdown_tx, _) = tokio::sync::watch::channel(false);
    let shutdown_state = Arc::clone(&state);
    let drain_conn_manager = Arc::clone(&state.conn_manager);
    let drain_jitter_ms = state.config.drain_jitter_ms;
    let tx = shutdown_tx.clone();
    // TODO(coverage): `serve`'s shutdown wiring has no automated test. The
    // jittered drain helper (`ConnectionManager::drain_all_jittered`) is
    // covered in `state.rs`, but coverage of the helper is not coverage of
    // its use here: the three wiring facts below are currently unguarded, and
    // mutating any one of them leaves the suite green.
    //   1. Jitter dispatch: `drain_jitter_ms == 0` must pick `drain_all`, and
    //      a non-zero value must pick `drain_all_jittered(drain_jitter_ms)`.
    //      A mutant that inverts this condition ships jitter-off in prod.

View on GitHub (pinned to ef2aa1ae38)