What causes "queue set already closed" in slackhq/nebula?

Check queue set state (or a exposed closed flag) before calling Add; skip adding fds for a closed set.

queue set already closed

Error message

queue set already closed

What it means

The GSO/offload queue set (offloadQueueSet) returns a fresh errors.New("queue set already closed") from Add when its closed atomic flag is set, meaning the queue set has been shut down and no new file descriptors may be attached as offload queues. It is a lifecycle guard preventing adds after Close.

Source

Thrown at overlay/tio/queueset_gso_linux.go:54

	out := &offloadQueueSet{
		pq:         []*Offload{},
		pqi:        []Queue{},
		shutdownFd: shutdownFd,
		usoEnabled: usoEnabled,
		l:          l,
	}

	return out, nil
}

func (c *offloadQueueSet) Queues() []Queue {
	return c.pqi
}

func (c *offloadQueueSet) Add(fd int) error {
	if c.closed.Load() {
		return errors.New("queue set already closed")
	}
	x, err := newOffload(fd, c.shutdownFd, c.usoEnabled, c.l)
	if err != nil {
		return err
	}
	c.pq = append(c.pq, x)
	c.pqi = append(c.pqi, x)

	return nil
}

func (c *offloadQueueSet) wakeForShutdown() error {
	var buf [8]byte
	binary.NativeEndian.PutUint64(buf[:], 1)
	_, err := unix.Write(c.shutdownFd, buf[:])
	return err
}

View on GitHub (pinned to dd8f660c0a)

Solutions

Check queue set state (or a exposed closed flag) before calling Add; skip adding fds for a closed set.
Serialize shutdown and fd registration so Add cannot run concurrently with Close (mutex or single-owner goroutine).
Make Close wait for in-flight Add calls (WaitGroup) so the race window disappears.
Handle the error gracefully in the caller: drop/close the fd, since the queue set is going away anyway.
Aggregate a sentinel error so callers can errors.Is-compare instead of string matching.

Example fix

// before: string-matching an ad-hoc error
if err := qs.Add(fd); err != nil {
    if err.Error() == "queue set already closed" { ... }
}
// after: check state before adding
if qs.Closed() {
    fd.Close()
    return
}
if err := qs.Add(fd); err != nil {
    fd.Close()
}

Defensive patterns

Strategy: try-catch

Validate before calling

// Go: check lifecycle state before Add
if qs.Closed() {
    fd.Close()
    return
}

Try / catch

if err := qs.Add(fd); err != nil {
    if strings.Contains(err.Error(), "already closed") || qs.Closed() {
        fd.Close()
        return nil // benign during shutdown
    }
    return err
}

Prevention

Expose an IsClosed()/Closed() check and consult it before every Add.
Use a sentinel error (var ErrClosed = errors.New(...)) so errors.Is works reliably.
Synchronize Add and Close with a mutex or WaitGroup to eliminate the race.
During shutdown, close fds in the Add caller rather than registering them.
Log shutdown races explicitly to find accept-loop/teardown ordering bugs.

When it happens

Trigger: Calling offloadQueueSet.Add(fd) after the queue set's Close/shutdown path has flipped c.closed. Typical when socket registration races with shutdown: a new connection's fd is handed to Add while another goroutine is tearing the queue set down.

Common situations: Server shutdown while connections are still being accepted; listener reconfiguration that closes and recreates queue sets; fd registration racing with process-level teardown; test harnesses closing the set before draining pending adds.

Related errors

AI-assisted analysis of slackhq/nebula@dd8f660c0a (2026-09-03). Data as JSON: /api/errors/e9e15eb4e38a47b1. Report an issue: GitHub.