nsqio/nsq · critical

listener.Accept() error - %s

Error message

listener.Accept() error - %s

What it means

internal/protocol.TCPServer (internal/protocol/tcp_server.go) is the accept loop behind nsqd's 4150 and nsqlookupd's 4160 TCP ports. Temporary accept errors (net.Error.Temporary) are logged as warnings and retried after a Gosched; a 403-like ErrClosed from listener.Close during shutdown breaks the loop cleanly. Any other accept error is wrapped as 'listener.Accept() error - %s' and returned, terminating the TCP serving goroutine and, via main's error handling, the process.

Source

Thrown at internal/protocol/tcp_server.go:34

func TCPServer(listener net.Listener, handler TCPHandler, logf lg.AppLogFunc) error {
	logf(lg.INFO, "TCP: listening on %s", listener.Addr())

	var wg sync.WaitGroup

	for {
		clientConn, err := listener.Accept()
		if err != nil {
			// net.Error.Temporary() is deprecated, but is valid for accept
			// this is a hack to avoid a staticcheck error
			if te, ok := err.(interface{ Temporary() bool }); ok && te.Temporary() {
				logf(lg.WARN, "temporary Accept() failure - %s", err)
				runtime.Gosched()
				continue
			}
			// theres no direct way to detect this error because it is not exposed
			if !errors.Is(err, net.ErrClosed) {
				return fmt.Errorf("listener.Accept() error - %s", err)
			}
			break
		}

		wg.Add(1)
		go func() {
			handler.Handle(clientConn)
			wg.Done()
		}()
	}

	// wait to return until all handler goroutines complete
	wg.Wait()

	logf(lg.INFO, "TCP: closing %s", listener.Addr())

	return nil
}

View on GitHub (pinned to 85cf10c09c)

Solutions

  1. Read the wrapped syscall text: 'too many open files' -> raise ulimit -n / LimitNOFILE and check /proc/<pid>/fd for leaks; other errnos point to the socket manager.
  2. Restart the process once limits are fixed; the error is fatal to the daemon by design.
  3. Audit for fd leaks: 'ls /proc/<pid>/fd | wc -l' over time; ensure clients (especially broken custom ones) close connections.
  4. If using socket activation/proxies, test plain binding first to rule out fd handoff problems.

Example fix

# before
# nsqd: listener.Accept() error - accept tcp 0.0.0.0:4150: accept4: too many open files
ulimit -n 1024

# after
# raise limits and restart
ulimit -n 65536   # or systemd LimitNOFILE=65536
systemctl restart nsqd
Defensive patterns

Strategy: retry

Validate before calling

// pre-flight: ensure the TCP port is bindable before daemon start
ln, err := net.Listen("tcp", addr)
if err != nil {
    return fmt.Errorf("cannot bind %s: %w", addr, err)
}
ln.Close() // release; the daemon binds next

Try / catch

// TCPServer's return is fatal by design; supervisors should catch, classify, restart
if err := protocol.TCPServer(ln, &prot); err != nil && !errors.Is(err, net.ErrClosed) {
    log.Printf("tcp accept loop died: %v", err)
    if strings.Contains(err.Error(), "too many open files") {
        bumpFDLimitThenRestart()
    } else {
        restart()
    }
}

Prevention

When it happens

Trigger: Accept() failing with a non-temporary, non-ErrClosed error: fd exhaustion that the runtime no longer classifies as temporary, ENFILE/ENOMEM system-wide, a listener fd invalidated underneath the process (hot socket transfer done wrong, container socket passthrough), or EBADF after a buggy external Close. Deliberate shutdown does NOT trigger this — Close produces net.ErrClosed which is filtered.

Common situations: Hosts exhausting file descriptors because of connection churn or leaked conns from misbehaving clients; exotic socket managers (systemd socket activation with wrong settings, sidecar proxies) handing over bad fds; OS-level limits (fs.file-max) hit during traffic spikes; running under old runtimes with different Temporary() semantics.

Related errors

AI-assisted analysis of nsqio/nsq@85cf10c09c (2026-08-16). Data as JSON: /api/errors/60ba241a6de4af8d. Report an issue: GitHub.