hibiken/asynq · error

nodes not found

Error message

nodes not found

What it means

Returned by RDB cluster-slot lookup when ClusterKeySlot computed a hash slot for a queue but the ClusterSlots() response contained no range covering that slot. This should be impossible with a healthy Redis Cluster (all 16384 slots are owned), so it indicates an incomplete cluster or a truncated slot map.

Solutions

  1. Run CLUSTER SLOTS / redis-cli --cluster check to find unassigned slots
  2. Wait for resharding/failover to complete and retry
  3. Assign the missing slots: redis-cli --cluster fix
  4. Ensure the cluster owns all 16384 slots before relying on slot lookups

Example fix

// before
nodes, err := r.clusterNodeLookup(qname)
if err != nil { return err }
// after
nodes, err := r.clusterNodeLookup(qname)
if err != nil {
    time.Sleep(500 * time.Millisecond) // transient during reshard
    nodes, err = r.clusterNodeLookup(qname)
}
if err != nil { return err }
Defensive patterns

Strategy: retry

Validate before calling

slots, err := client.ClusterSlots(ctx).Result()
covered := false
for _, s := range slots { if s.Start <= slot && slot <= s.End { covered = true } }
if !covered { return errors.New("cluster has unassigned slots") }

Try / catch

var nodes []SlotRangeNode
err := retry.Do(func() error {
    var e error
    nodes, e = inspector clusterLookup(qname)
    return e
}, retry.Attempts(3), retry.Delay(500*time.Millisecond))

Prevention

When it happens

Trigger: Calling ClusterKeySlot/cluster-node resolution against a Redis Cluster that has unassigned slots (during resharding or failover), or a ClusterSlots reply missing the relevant slot range.

Common situations: Cluster resharding in progress; a failing primary whose slots are not yet migrated; misconfigured cluster with fewer nodes than slots; running cluster-mode code against a non-cluster setup.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors


AI-assisted analysis of hibiken/asynq@d135f1439b (2026-09-07). Data as JSON: /api/errors/18094976ef0db5f2. Report an issue: GitHub.

Appendix: source

Thrown at internal/rdb/inspect.go:2121

	return r.client.ClusterKeySlot(context.Background(), key).Result()
}

// ClusterNodes returns a list of nodes the given queue belongs to.
func (r *RDB) ClusterNodes(qname string) ([]redis.ClusterNode, error) {
	keyslot, err := r.ClusterKeySlot(qname)
	if err != nil {
		return nil, err
	}
	clusterSlots, err := r.client.ClusterSlots(context.Background()).Result()
	if err != nil {
		return nil, err
	}
	for _, slotRange := range clusterSlots {
		if int64(slotRange.Start) <= keyslot && keyslot <= int64(slotRange.End) {
			return slotRange.Nodes, nil
		}
	}
	return nil, fmt.Errorf("nodes not found")
}

View on GitHub (pinned to d135f1439b)