handoff queue is full, cannot queue new handoff requests - c

Error message

handoff queue is full, cannot queue new handoff requests - consider increasing HandoffQueueSize or MaxWorkers in configuration

What it means

Returned by queueHandoff (handoff_worker.go:326) when the handoff queue channel is full after waiting up to 100ms for a slot (handoff_worker.go:313-321). This means pending connection handoffs are accumulating faster than the worker pool can drain them; the offending connection is removed from the pool rather than queued (pool_hook.go:153-157).

Source

Thrown at maintnotifications/errors.go:35

	ErrInvalidMaintNotifications         = errors.New(logs.InvalidMaintNotificationsError())
	ErrMaxHandoffRetriesReached          = errors.New(logs.MaxHandoffRetriesReachedError())

	// Configuration validation errors

	// ErrInvalidHandoffRetries is returned when the number of handoff retries is invalid
	ErrInvalidHandoffRetries = errors.New(logs.InvalidHandoffRetriesError())
)

// Integration errors
var (
	// ErrInvalidClient is returned when the client does not support push notifications
	ErrInvalidClient = errors.New(logs.InvalidClientError())
)

// Handoff errors
var (
	// ErrHandoffQueueFull is returned when the handoff queue is full
	ErrHandoffQueueFull = errors.New(logs.HandoffQueueFullError())
)

// Notification errors
var (
	// ErrInvalidNotification is returned when a notification is in an invalid format
	ErrInvalidNotification = errors.New(logs.InvalidNotificationError())
)

// connection handoff errors
var (
	// ErrConnectionMarkedForHandoff is returned when a connection is marked for handoff
	// and should not be used until the handoff is complete
	ErrConnectionMarkedForHandoff = errors.New(logs.ConnectionMarkedForHandoffErrorMessage)
	// ErrConnectionMarkedForHandoffWithState is returned when a connection is marked for handoff
	// and should not be used until the handoff is complete
	ErrConnectionMarkedForHandoffWithState = errors.New(logs.ConnectionMarkedForHandoffErrorMessage + " with state")
	// ErrConnectionInvalidHandoffState is returned when a connection is in an invalid state for handoff
	ErrConnectionInvalidHandoffState = errors.New(logs.ConnectionInvalidHandoffStateErrorMessage)

View on GitHub (pinned to 36d97525cd)

Solutions

Increase HandoffQueueSize (explicit minimum 200) and/or MaxWorkers to match expected burst volume.
Raise PoolSize — both MaxWorkers and HandoffQueueSize auto-scale from it.
Investigate why workers are slow: unreachable new endpoints cause each worker to block for HandoffTimeout, starving the queue. Fix endpoint routing / EndpointType.
Monitor via GetCircuitBreakerStats and pool metrics; if the circuit breaker is opening, the upstream is unhealthy — resolve that root cause.

Example fix

// before
cfg := &maintnotifications.Config{MaxWorkers: 1, HandoffQueueSize: 5}

// after
cfg := maintnotifications.DefaultConfig()
cfg.MaxWorkers = 20
cfg.HandoffQueueSize = 1000

Defensive patterns

Strategy: retry

Try / catch

if errors.Is(err, maintnotifications.ErrHandoffQueueFull) {
    // backpressure: connection dropped, pool will use others. 
    // Retry the command; if persistent, raise HandoffQueueSize/MaxWorkers/PoolSize.
}

Prevention

Size HandoffQueueSize and MaxWorkers for migration bursts, not steady state.
Scale PoolSize up — both auto-scale from it.
Fix unreachable endpoints so workers aren't blocked on HandoffTimeout.
Monitor circuit-breaker and queue stats.

When it happens

Trigger: A burst of MOVING notifications (e.g. a large cluster rebalancing or multi-node failover) marks many connections for handoff simultaneously while MaxWorkers is low and/or HandoffQueueSize is undersized. OnPut tries to queue the handoff, the channel is full, ensureWorkerAvailable is already at maxWorkers, and the 100ms grace window expires.

Common situations: Small PoolSize driving a tiny auto-calculated queue; a migration storm; workers stuck on slow dials (HandoffTimeout long, endpoint unreachable) so they never free up to drain the queue.

Related errors

AI-assisted analysis of go-redis/redis@36d97525cd (2026-08-06). Data as JSON: /data/errors/79c8b81ad6561e4a.json. Report an issue: GitHub.