kubernetes/kops · critical
error listening on %q: %v
Error message
error listening on %q: %v
What it means
The healthcheck server fails when http.ListenAndServe cannot bind/serve on the configured --listen address. Common causes are the port already in use, an invalid address, or insufficient privileges for the port. The wrapped error includes the listen address and OS error.
Source
Thrown at cmd/kube-apiserver-healthcheck/main.go:185
}
tlsConfig.Certificates = []tls.Certificate{keypair}
}
transport := &http.Transport{
TLSClientConfig: tlsConfig,
}
s := &healthCheckServer{
transport: transport,
}
http.HandleFunc("/", s.handler)
klog.Infof("listening on %s", listen)
if err := http.ListenAndServe(listen, nil); err != nil {
return fmt.Errorf("error listening on %q: %v", listen, err)
}
return fmt.Errorf("unexpected return from ListenAndServe")
}
func main() {
if err := run(); err != nil {
fmt.Fprintf(os.Stderr, "%v\n", err)
os.Exit(1)
}
}
View on GitHub (pinned to 4c8573c808)
Solutions
- Check for a port conflict: ss -ltnp / lsof -i :<port> and kill the conflicting process or change --listen.
- Use a non-privileged port (e.g. 3990) or grant the bind capability (setcap / securityContext).
- Verify the --listen address syntax is valid host:port.
- Ensure only one healthcheck instance runs per pod/netns.
Example fix
// before
run("--listen", "0.0.0.0:443", ...) // as non-root
// after
run("--listen", "0.0.0.0:3990", ...) Defensive patterns
Strategy: validation
Validate before calling
// check the port is free before binding
ln, err := net.Listen("tcp", listen)
if err != nil {
return fmt.Errorf("cannot bind %s: %w", listen, err)
}
ln.Close() Try / catch
// Go: handle bind failure with a clear operator message
if err := run(); err != nil {
if strings.Contains(err.Error(), "error listening") {
klog.Fatalf("port conflict or bad address: %v", err)
}
} Prevention
- Use an unprivileged port (e.g. 3990) for the healthcheck sidecar.
- Detect port conflicts with ss/lsof before deploying.
- Never run two instances with the same --listen in one netns.
- Validate the --listen address format in config templating.
When it happens
Trigger: run() calls http.ListenAndServe(listen, nil) and the OS returns e.g. 'address already in use', 'permission denied' (port <1024 as non-root), or 'invalid address'.
Common situations: Another process (or a previous unclean pod) holds the port; two sidecars with the same --listen; binding :443 without NET_BIND_SERVICE; malformed host:port string.
Related errors
- unexpected return from ListenAndServe
- serving readiness probe: %w
- error querying kubernetes version: %v
- error loading channel %q: %v
- error querying ec2 metadata service (for region): %v
AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05).
Data as JSON: /api/errors/913d2a1b53d01e7e.
Report an issue: GitHub.