cloudflare/pingora · critical

Failed to build listeners

Error message

Failed to build listeners

What it means

When a listening service starts, pingora binds every configured endpoint via listeners.build(fds) and expects success. build() creates the actual TCP/UDS listeners (TransportStackBuilder::build -> builder.listen), adopts systemd-upgrade fds when present, and constructs TLS settings. Any failure — address already in use, permission denied on privileged ports, bad unix socket path, or invalid TLS settings — aborts the whole service startup with this message.

Source

Thrown at pingora-core/src/services/listening.rs:293

}

#[async_trait]
impl<A: ServerApp + Send + Sync + 'static> ServiceTrait for Service<A> {
    async fn start_service(
        &mut self,
        #[cfg(unix)] fds: Option<ListenFds>,
        shutdown: ShutdownWatch,
        listeners_per_fd: usize,
    ) {
        let runtime = current_handle();
        let endpoints = self
            .listeners
            .build(
                #[cfg(unix)]
                fds,
            )
            .await
            .expect("Failed to build listeners");

        let app_logic = self
            .app_logic
            .take()
            .expect("can only start_service() once");
        let app_logic = Arc::new(app_logic);

        let mut handlers = Vec::new();

        endpoints.into_iter().for_each(|endpoint| {
            for _ in 0..listeners_per_fd {
                let shutdown = shutdown.clone();
                let my_app_logic = app_logic.clone();
                let endpoint = endpoint.clone();

                let jh = runtime.spawn(async move {
                    Self::run_endpoint(my_app_logic, endpoint, shutdown).await;
                });

View on GitHub (pinned to 0046038bd4)

Solutions

1. Find what holds the address: `ss -ltnp | grep <port>` or `lsof -i :<port>`; stop the old process or change the bind address/port
2. For privileged ports grant CAP_NET_BIND_SERVICE (`setcap cap_net_bind_service=+ep ./binary`) or use a front proxy / net.ipv4.ip_unprivileged_port_start
3. Verify every TLS cert/key path is absolute, readable by the service user, and parses (`openssl x509 -in cert -noout`, `openssl pkey -in key -noout`)
4. Remove stale unix socket files, or ensure the parent directory is writable by the service user
5. With systemd socket activation, make sure the number of passed fds matches the configured listeners

Example fix

# before: privileged port without capability, or port already in use
./my_proxy --daemon -d 0.0.0.0:80,0.0.0.0:443   # panics: Failed to build listeners

# after: grant bind capability (or move to an unprivileged port)
sudo setcap cap_net_bind_service=+ep ./my_proxy
./my_proxy --daemon -d 0.0.0.0:80,0.0.0.0:443

use std::net::TcpListener;

// Validate at config-load time, before run_service()
fn endpoints_bindable(addrs: &[std::net::SocketAddr]) -> bool {
    addrs.iter().all(|a| TcpListener::bind(a).is_ok())
}

// Also pre-parse TLS material so bad certs fail loudly, not in listeners.build()
fn tls_material_ok(cert: &std::path::Path, key: &std::path::Path) -> bool {
    std::process::Command::new("openssl")
        .args(["x509", "-in"]).arg(cert).arg("-noout").status().map(|s| s.success()).unwrap_or(false)
    // repeat for the key with `openssl pkey -in key -noout`
}

When it happens

Trigger: run_service()/service start with: a bind port already taken; binding port <1024 without CAP_NET_BIND_SERVICE; a unix socket path that exists or sits in a non-writable directory; TLS endpoints with invalid/unreadable cert or key material; zero-cost upgrade path where passed ListenFds don't match configured listeners.

Common situations: An old process or second instance still holding the port; docker port-publish conflicts; missing CAP_NET_BIND_SERVICE in containers for 80/443; relative cert paths breaking when cwd changes under systemd; stale socket files left from a crash.

Related errors

• Tried to listen with no addr specified
• Failed to parse certificate from DER format.
• Failed to parse certificate from DER format.
• Failed to parse PEM
• failed to build work-stealing Tokio runtime

AI-assisted analysis of cloudflare/pingora@0046038bd4 (2026-08-16). Data as JSON: /api/errors/dc39802513d6b10f. Report an issue: GitHub.