slackhq/nebula · error

queue set already closed

Error message

queue set already closed

What it means

pollQueueSet (epoll/poll-based queue set) returns errors.New("queue set already closed") from Add when its closed atomic flag indicates the set has been shut down, blocking registration of new file descriptors into a dead poll set. It is the poll-mode analogue of the GSO queue-set guard.

Source

Thrown at overlay/tio/queueset_poll_linux.go:44

		return nil, fmt.Errorf("failed to create eventfd: %w", err)
	}

	out := &pollQueueSet{
		pq:         []*Poll{},
		pqi:        []Queue{},
		shutdownFd: shutdownFd,
	}

	return out, nil
}

func (c *pollQueueSet) Queues() []Queue {
	return c.pqi
}

func (c *pollQueueSet) Add(fd int) error {
	if c.closed.Load() {
		return errors.New("queue set already closed")
	}
	x, err := newPoll(fd, c.shutdownFd)
	if err != nil {
		return err
	}
	c.pq = append(c.pq, x)
	c.pqi = append(c.pqi, x)

	return nil
}

func (c *pollQueueSet) wakeForShutdown() error {
	var buf [8]byte
	binary.NativeEndian.PutUint64(buf[:], 1)
	_, err := unix.Write(int(c.shutdownFd), buf[:])
	return err
}

View on GitHub (pinned to dd8f660c0a)

Solutions

1. Check closed state before Add and close the fd instead of registering it.
2. Guard the closed check and pq append with a mutex so Add is atomic with respect to Close.
3. Use a WaitGroup in Close so in-flight Adds complete before teardown finishes.
4. In callers, treat this error as benign during shutdown: release the fd and continue.
5. Introduce a package-level sentinel error for errors.Is comparisons and tests.

Example fix

// before: adding without state check
if err := pqs.Add(fd); err != nil {
    return err
}
// after: guard against closed set
select {
case <-pqs.Done():
    fd.Close()
    return nil
default:
}
if err := pqs.Add(fd); err != nil {
    fd.Close()
    return err
}

// Go: skip registration when the poll set is shutting down
if pqs.Closed() {
    fd.Close()
    return
}

if err := pqs.Add(fd); err != nil {
    if pqs.Closed() {
        fd.Close()
        return nil // shutdown in progress
    }
    return err
}

When it happens

Trigger: Calling pollQueueSet.Add(fd) after Close set c.closed. Occurs when a connection fd is registered for polling while the poll set is concurrently being torn down, or when stale references to a replaced queue set are used.

Common situations: Graceful shutdown racing with new connection registration; hot-reload of listeners swapping queue sets; a deferred Add running after an early return path closed the set; tests closing the set before all fds are registered.

Related errors

• queue set already closed
• ErrUnknownState
• errPacketTooShort
• packet is too short
• failed to enable offload on multiqueue tun fd: %w

AI-assisted analysis of slackhq/nebula@dd8f660c0a (2026-09-03). Data as JSON: /api/errors/a420d7c27bd58b83. Report an issue: GitHub.