nsqio/nsq · critical

http.Serve() error - %s

Error message

http.Serve() error - %s

What it means

internal/http_api.Serve runs the HTTP accept loop for nsqd's, nsqlookupd's and nsqadmin's web listeners (http_server.go). It calls http.Server.Serve on the bound listener and treats net.ErrClosed as the normal shutdown signal; any other error from Serve is wrapped as 'http.Serve() error - %s' and returned, which propagates to main and exits the process. In practice Serve only errors on listener-level failures, since per-connection handler errors stay inside the server.

Source

Thrown at internal/http_api/http_server.go:32

	logf lg.AppLogFunc
}

func (l logWriter) Write(p []byte) (int, error) {
	l.logf(lg.WARN, "%s", string(p))
	return len(p), nil
}

func Serve(listener net.Listener, handler http.Handler, proto string, logf lg.AppLogFunc) error {
	logf(lg.INFO, "%s: listening on %s", proto, listener.Addr())

	server := &http.Server{
		Handler:  handler,
		ErrorLog: log.New(logWriter{logf}, "", 0),
	}
	err := server.Serve(listener)
	// theres no direct way to detect this error because it is not exposed
	if err != nil && !errors.Is(err, net.ErrClosed) {
		return fmt.Errorf("http.Serve() error - %s", err)
	}

	logf(lg.INFO, "%s: closing %s", proto, listener.Addr())

	return nil
}

View on GitHub (pinned to 85cf10c09c)

Solutions

1. Check fd limits and usage on the host: 'ulimit -n', 'ls /proc/$(pgrep nsqd)/fd | wc -l'; raise limits (systemd LimitNOFILE) if near the cap.
2. Look at the wrapped %s text — it names the true cause; act on that syscall error rather than the wrapper.
3. Restart the daemon; this is a fatal path by design and the data files are safe (nsqd persists its queue).
4. If it recurs, capture 'dmesg' and Go stack traces, and verify you are not running a very old binary against a newer kernel/socket setup.

Example fix

# before: fd exhaustion surfaces as fatal Serve error at some point
ulimit -n 1024
# nsqd: http.Serve() error - accept tcp ...: too many open files

# after
# /etc/systemd/system/nsqd.service
[Service]
LimitNOFILE=65536
# then: systemctl daemon-reload && systemctl restart nsqd

// before binding, confirm fd headroom so Serve is unlikely to die of EMFILE
var st syscall.Statfs_t
_ = syscall.Statfs("/", &st)
soft, hard := ulimits() // syscall.Getrlimit(RLIMIT_NOFILE)
if used := countOpenFDs(); soft-used < 1000 {
    warn("raise RLIMIT_NOFILE before serving HTTP")
}

// supervisors: any non-nil return from http_api.Serve is fatal for the daemon;
// catch, log the wrapped cause, and restart with alerting
if err := http_api.Serve(ln, mux, "HTTP", logf); err != nil && !errors.Is(err, net.ErrClosed) {
    log.Printf("fatal: %v — restarting", err)
    notifyOnCall(err)
    restart()
}

When it happens

Trigger: The listener's fd breaks or cannot accept anymore without being explicitly Closed: file-descriptor exhaustion at the OS level surfacing as a non-temporary accept error, a listener closed via an unexpected code path that does not map to net.ErrClosed, or (rare) kernel/driver errors on accept. Normal Ctrl-C/SIGTERM shutdown paths Close the listener and are filtered out, so they do not produce this error.

Common situations: Long-running daemons on hosts with tiny ulimit -n where EMFILE eventually becomes non-temporary; container runtimes reaping sockets oddly; custom wrappers closing the listener directly instead of via the provided shutdown; upgrades of Go changing error wrapping so an older binary's is-net.ErrClosed check misses (modern nsq versions match correctly).

Related errors

• listener.Accept() error - %s
• E_REQ_FAILED
• E_SUB_FAILED
• E_INVALID
• exiting

AI-assisted analysis of nsqio/nsq@85cf10c09c (2026-08-16). Data as JSON: /api/errors/419b9c40e6b27e51. Report an issue: GitHub.