thanos-io/thanos · error
error starting web server
Error message
error starting web server
What it means
runReceive launches the receive HTTP web handler (write path, /api/v1/receive, etc.) in its own goroutine via errgroup. The handler's blocking Run() error (bind failure, listener error, runtime failure) is wrapped with 'error starting web server', which then tears the whole receive process down.
Solutions
- Check the wrapped inner error: 'address already in use' means find and stop the conflicting process or change --receive.http-address.
- Use ss -ltnp / netstat -ltnp to see what holds the port.
- Use a non-privileged port (>1024) or grant the binary NET_BIND_SERVICE capability.
- Verify the listen address syntax and that it exists on the host (e.g. 0.0.0.0 vs a specific IP, IPv6 support).
- Inspect receive logs for why the web handler failed after startup.
Example fix
// before thanos receive --receive.http-address=:19191 # port already taken // after thanos receive --receive.http-address=:10902
Defensive patterns
Strategy: validation
Validate before calling
ln, err := net.Listen("tcp", httpAddr)
if err != nil { return fmt.Errorf("cannot bind %s: %w", httpAddr, err) }
ln.Close() // bind check before starting thanos Try / catch
if err := run(); err != nil {
if strings.Contains(err.Error(), "error starting web server") {
if strings.Contains(err.Error(), "address already in use") {
log.Errorf("port conflict on %s; change --receive.http-address", httpAddr)
}
}
} Prevention
- Pre-check port availability in entrypoint scripts
- Use non-privileged ports (>1024) or NET_BIND_SERVICE capability
- Assign unique ports per replica when co-locating processes
- Validate listen address syntax and IP family before deployment
When it happens
Trigger: webHandler.Run() fails, most commonly because the configured listen address (--receive.http-address / rw address) is already in use, the address/port is invalid, or the listener errors at runtime.
Common situations: Another process (or a second receive replica on the same host) already bound to the port; port 80/1024 restricted for non-root; IPv6 address configured on an IPv4-only host; typo in the address string.
Related errors
- failed to validate prometheus flags
- no query API server reachable
- initial external labels query
- error sending proto response
- send request to
AI-assisted analysis of thanos-io/thanos@35b8b99117 (2026-09-07).
Data as JSON: /api/errors/40499df2c320b3d7.
Report an issue: GitHub.
Appendix: source
Thrown at cmd/thanos/receive.go:495
func() error {
level.Info(logger).Log("msg", "listening for StoreAPI and WritableStoreAPI gRPC", "address", conf.grpcConfig.bindAddress)
statusProber.Healthy()
return srv.ListenAndServe()
},
func(err error) {
statusProber.NotReady(err)
defer statusProber.NotHealthy(err)
srv.Shutdown(err)
},
)
}
level.Debug(logger).Log("msg", "setting up receive HTTP handler")
{
g.Add(
func() error {
return errors.Wrap(webHandler.Run(), "error starting web server")
},
func(err error) {
webHandler.Close()
},
)
}
if limitsConfig.AreHeadSeriesLimitsConfigured() {
level.Info(logger).Log("msg", "setting up periodic (every 15s) meta-monitoring query for limiting cache")
{
ctx, cancel := context.WithCancel(context.Background())
g.Add(func() error {
return runutil.Repeat(15*time.Second, ctx.Done(), func() error {
if err := limiter.HeadSeriesLimiter().QueryMetaMonitoring(ctx); err != nil {
level.Error(logger).Log("msg", "failed to query meta-monitoring", "err", err.Error())
}
return nil
})View on GitHub (pinned to 35b8b99117)