{"record":{"id":"d3ce7333e79e68b6","repo":"block/buzz","slug":"failed-to-bind-health-port-e","errorCode":null,"errorMessage":"Failed to bind health port {}: {e}","messagePattern":"Failed to bind health port (.+?): (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"critical","filePath":"crates/buzz-relay/src/main.rs","lineNumber":1373,"sourceCode":"/// `state.rs`) sum to 25s and stay inside the 30s hard drain. Total worst\n/// case from SIGTERM to forced exit is 5s + 30s = 35s. Both fit inside the\n/// chart's `terminationGracePeriodSeconds: 60` (`deploy/charts/buzz/values.yaml`),\n/// which leaves headroom but assumes no `preStop` hook adds further delay.\n/// With jitter off (`BUZZ_DRAIN_JITTER_MS=0`, the default) sockets close\n/// all-at-once right after the grace, so the per-socket delay collapses to\n/// roughly the 5s grace plus the ack wait.\nconst GRACEFUL_DRAIN_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(30);\n\nasync fn serve(\n    router: axum::Router,\n    health_router: axum::Router,\n    state: Arc<AppState>,\n) -> anyhow::Result<()> {\n    let config = &state.config;\n\n    let health_listener = tokio::net::TcpListener::bind((\"0.0.0.0\", config.health_port))\n        .await\n        .map_err(|e| anyhow::anyhow!(\"Failed to bind health port {}: {e}\", config.health_port))?;\n    info!(port = config.health_port, \"Health probe listener started\");\n    tokio::spawn(async move {\n        axum::serve(health_listener, health_router).await.ok();\n    });\n\n    let (shutdown_tx, _) = tokio::sync::watch::channel(false);\n    let shutdown_state = Arc::clone(&state);\n    let drain_conn_manager = Arc::clone(&state.conn_manager);\n    let drain_jitter_ms = state.config.drain_jitter_ms;\n    let tx = shutdown_tx.clone();\n    // TODO(coverage): `serve`'s shutdown wiring has no automated test. The\n    // jittered drain helper (`ConnectionManager::drain_all_jittered`) is\n    // covered in `state.rs`, but coverage of the helper is not coverage of\n    // its use here: the three wiring facts below are currently unguarded, and\n    // mutating any one of them leaves the suite green.\n    //   1. Jitter dispatch: `drain_jitter_ms == 0` must pick `drain_all`, and\n    //      a non-zero value must pick `drain_all_jittered(drain_jitter_ms)`.\n    //      A mutant that inverts this condition ships jitter-off in prod.","sourceCodeStart":1355,"sourceCodeEnd":1391,"githubUrl":"https://github.com/block/buzz/blob/ef2aa1ae38fadcc0bc22b8bf6ed96b35933146be/crates/buzz-relay/src/main.rs#L1355-L1391","documentation":"buzz-relay's serve() binds a dedicated TCP listener on 0.0.0.0:$BUZZ_HEALTH_PORT for health probes before anything else starts. If the kernel refuses the bind (AddrInUse, PermissionDenied, EADDRNOTAVAIL), the map_err wraps the io::Error into anyhow with the port number and the whole process exits at boot. This is a fail-fast startup check: a relay without a reachable health port would be killed by orchestrators anyway.","triggerScenarios":"TcpListener::bind((\"0.0.0.0\", config.health_port)) at crates/buzz-relay/src/main.rs:1255 fails when: (1) another process (often a second relay instance or a leftover container) already listens on BUZZ_HEALTH_PORT; (2) BUZZ_HEALTH_PORT is set below 1024 and the process lacks CAP_NET_BIND_SERVICE; (3) BUZZ_HEALTH_PORT collides with the relay's own bind_addr port; (4) the host has no IPv4 interface (bind on 0.0.0.0 fails with EADDRNOTAVAIL).","commonSituations":"Running `just relay` twice in different terminals; a Docker/K8s pod where the health port is also declared in the container spec and a sidecar grabbed it; copying .env from a teammate whose BUZZ_HEALTH_PORT is already taken on the shared dev box; setting BUZZ_HEALTH_PORT=80 in docker-compose without root.","solutions":["Find and stop the occupant: `ss -ltnp 'sport = :<port>'` (or `lsof -i :<port>`), then kill that process or use another port.","Set BUZZ_HEALTH_PORT to a free high port in .env (e.g. 3001) and restart the relay.","If the port must be <1024, run the relay with CAP_NET_BIND_SERVICE (`setcap cap_net_bind_service=+ep` on the binary) or bind a high port and map it at the load balancer.","Make sure BUZZ_HEALTH_PORT differs from the main relay bind port (BUZZ_BIND_ADDR) and the mesh bind port.","In containers, ensure only one process binds the health port; remove duplicate port declarations from the pod spec."],"exampleFix":"# before: BUZZ_HEALTH_PORT=3000 already used by a stale relay\nss -ltnp 'sport = :3000'   # shows pid of old process\nkill <pid>\n\n# after: or just pick a free port in .env\nBUZZ_HEALTH_PORT=3001","handlingStrategy":"validation","validationCode":"# before starting the relay, confirm the health port is free\nif ss -ltnH 'sport = :'\"$BUZZ_HEALTH_PORT\" | grep -q .; then\n  echo \"health port $BUZZ_HEALTH_PORT already in use\" >&2; exit 1\nfi","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Allocate distinct ports for bind_addr, health_port, and mesh bind in .env and document them in the deployment values.","Prefer ephemeral/high ports (>1024) for BUZZ_HEALTH_PORT to avoid capability issues.","In systemd/K8s, have the unit/pod own the health port exclusively; never share it with sidecars.","Add a startup preflight (port-free check) to deploy scripts so failures are diagnosed before the relay binary runs."],"tags":["rust","network","port-bind","startup","health-check"],"backgroundTag":"port-already-in-use","analyzedSha":"ef2aa1ae38fadcc0bc22b8bf6ed96b35933146be","analyzedAt":"2026-08-20T04:38:24.874Z","contentChangedAt":"2026-08-20T04:38:24.874Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}