hibiken/asynq · error
nodes not found
Error message
nodes not found
What it means
Returned by RDB cluster-slot lookup when ClusterKeySlot computed a hash slot for a queue but the ClusterSlots() response contained no range covering that slot. This should be impossible with a healthy Redis Cluster (all 16384 slots are owned), so it indicates an incomplete cluster or a truncated slot map.
Solutions
- Run CLUSTER SLOTS / redis-cli --cluster check to find unassigned slots
- Wait for resharding/failover to complete and retry
- Assign the missing slots: redis-cli --cluster fix
- Ensure the cluster owns all 16384 slots before relying on slot lookups
Example fix
// before
nodes, err := r.clusterNodeLookup(qname)
if err != nil { return err }
// after
nodes, err := r.clusterNodeLookup(qname)
if err != nil {
time.Sleep(500 * time.Millisecond) // transient during reshard
nodes, err = r.clusterNodeLookup(qname)
}
if err != nil { return err } Defensive patterns
Strategy: retry
Validate before calling
slots, err := client.ClusterSlots(ctx).Result()
covered := false
for _, s := range slots { if s.Start <= slot && slot <= s.End { covered = true } }
if !covered { return errors.New("cluster has unassigned slots") } Try / catch
var nodes []SlotRangeNode
err := retry.Do(func() error {
var e error
nodes, e = inspector clusterLookup(qname)
return e
}, retry.Attempts(3), retry.Delay(500*time.Millisecond)) Prevention
- Run redis-cli --cluster check regularly to detect unassigned slots
- Avoid slot lookups during resharding windows
- Ensure all 16384 slots are assigned before serving traffic
- Alert on CLUSTER SLOTS gaps
When it happens
Trigger: Calling ClusterKeySlot/cluster-node resolution against a Redis Cluster that has unassigned slots (during resharding or failover), or a ClusterSlots reply missing the relevant slot range.
Common situations: Cluster resharding in progress; a failing primary whose slots are not yet migrated; misconfigured cluster with fewer nodes than slots; running cluster-mode code against a non-cluster setup.
Understand the failure class
Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.
Related errors
- task already exists
- task ID conflicts with another task
- queue not found
- queue is not empty
- task not found
AI-assisted analysis of hibiken/asynq@d135f1439b (2026-09-07).
Data as JSON: /api/errors/18094976ef0db5f2.
Report an issue: GitHub.
Appendix: source
Thrown at internal/rdb/inspect.go:2121
return r.client.ClusterKeySlot(context.Background(), key).Result()
}
// ClusterNodes returns a list of nodes the given queue belongs to.
func (r *RDB) ClusterNodes(qname string) ([]redis.ClusterNode, error) {
keyslot, err := r.ClusterKeySlot(qname)
if err != nil {
return nil, err
}
clusterSlots, err := r.client.ClusterSlots(context.Background()).Result()
if err != nil {
return nil, err
}
for _, slotRange := range clusterSlots {
if int64(slotRange.Start) <= keyslot && keyslot <= int64(slotRange.End) {
return slotRange.Nodes, nil
}
}
return nil, fmt.Errorf("nodes not found")
}
View on GitHub (pinned to d135f1439b)