block/buzz · error

Shutdown task failed: {e}

Error message

Shutdown task failed: {e}

What it means

serve() spawns a graceful-shutdown task (main.rs:1295): wait for SIGTERM/Ctrl+C, flip shutting_down, sleep a 5s grace, broadcast stop to all listeners, arm a 30s hard-exit timer (GRACEFUL_DRAIN_TIMEOUT), then drain every live WebSocket connection (jittered or all-at-once) and return the hard timer's abort handle. This error means shutdown_handle.await resolved to Err(JoinError): the task panicked — e.g. the .expect("install SIGTERM handler") on SignalKind::terminate(), or a bug inside the drain helpers — or it was cancelled, so the drain result and abort handle were lost. It is a bug/runtime indicator, not a configuration error. Only reachable on the UDS+TCP path (BUZZ_UDS_PATH set; uds_handle.abort() follows this line).

Source

Thrown at crates/buzz-relay/src/main.rs:1377

                })
                .await
                .ok();
        });

        let mut tcp_rx = shutdown_tx.subscribe();
        axum::serve(
            tcp_listener,
            router.into_make_service_with_connect_info::<std::net::SocketAddr>(),
        )
        .with_graceful_shutdown(async move {
            tcp_rx.changed().await.ok();
        })
        .await
        .map_err(|e| anyhow::anyhow!("TCP server error: {e}"))?;

        let hard_shutdown = shutdown_handle
            .await
            .map_err(|e| anyhow::anyhow!("Shutdown task failed: {e}"))?;
        uds_handle.abort();
        hard_shutdown.abort();
        return Ok(());
    }

    #[cfg(not(unix))]
    if config.uds_path.is_some() {
        tracing::warn!("BUZZ_UDS_PATH set but UDS not supported on this platform");
    }

    // TCP-only path.
    let mut tcp_rx = shutdown_tx.subscribe();
    axum::serve(
        tcp_listener,
        router.into_make_service_with_connect_info::<std::net::SocketAddr>(),
    )
    .with_graceful_shutdown(async move {
        tcp_rx.changed().await.ok();

View on GitHub (pinned to f956e6fe06)

Solutions

1. Scan upward in the logs for the panic message — JoinError carries the payload and it names the failing line (signal install vs drain).
2. Check how the binary is run: embedders must use a standard tokio runtime with unix signal support rather than a stripped-down runtime.
3. Update buzz-relay and report the panic payload — a panic in the shutdown path is a defect, not an operator error.
4. Operationally, rely on the supervisor's kill timeout (and the 30s hard-exit backstop if the task died after arming it) rather than expecting a clean drain.

fn is_task_panic(e: &tokio::task::JoinError) -> bool {
    e.is_panic()
}

fn is_task_cancelled(e: &tokio::task::JoinError) -> bool {
    e.is_cancelled()
}

match shutdown_handle.await {
    Ok(abort_handle) => {
        abort_handle.abort(); // drain finished; cancel the 30s hard-exit timer
    }
    Err(e) if e.is_panic() => {
        let payload = e.into_panic();
        tracing::error!(?payload, "shutdown task panicked — connections may not have drained");
        std::process::exit(1);
    }
    Err(e) => return Err(anyhow!("Shutdown task failed: {e}")),
}

When it happens

Trigger: A panic inside the shutdown task: installing the SIGTERM handler fails (runtime or sandbox without unix signal support, double handler registration); a panic in ConnectionManager::drain_all_jittered during the 1012-close sequence; an embedder aborting the task. Typically observed right after sending SIGTERM to a UDS-configured relay.

Common situations: Almost always a relay defect or an exotic embedding (custom tokio runtime without signal support). The process exits with this error instead of completing a clean drain; connections may not have received their 1012 close frames.

Related errors

• Failed to bind health port {}: {e}
• Failed to bind {}: {e}
• deletion heartbeat task failed: {error}
• git conformance probe failed: {e}
• BUZZ_UDS_PATH {uds_path} exists but is not a socket

AI-assisted analysis of block/buzz@f956e6fe06 (2026-08-16). Data as JSON: /api/errors/fb5f6ee6a6d906a5. Report an issue: GitHub.