weaviate/weaviate · error

unexpected added node %q is also in remaining for shard %q

Error message

unexpected added node %q is also in remaining for shard %q

What it means

A node marked as added to a shard also appears in the remaining set. A node cannot be both newly added and already present, so this contradictory diff aborts plan generation to prevent no-op or conflicting replication actions.

Source

Thrown at cluster/replication/manager.go:569

		scalePlan.ShardReplicationScaleActions[shardName] = cmd.ShardReplicationScaleActions{
			RemoveNodes: make(map[string]struct{}, len(nodes.Removed)),
			AddNodes:    make(map[string]string, len(nodes.Added)),
		}

		for node := range nodes.Removed {
			if _, ok := nodes.Remaining[node]; ok {
				return nil, fmt.Errorf("unexpected removed node %q is also in remaining for shard %q", node, shardName)
			}
			if _, ok := nodes.Added[node]; ok {
				return nil, fmt.Errorf("unexpected removed node %q is also in added for shard %q", node, shardName)
			}

			scalePlan.ShardReplicationScaleActions[shardName].RemoveNodes[node] = struct{}{}
		}

		for node := range nodes.Added {
			if _, ok := nodes.Remaining[node]; ok {
				return nil, fmt.Errorf("unexpected added node %q is also in remaining for shard %q", node, shardName)
			}
			if _, ok := nodes.Removed[node]; ok {
				return nil, fmt.Errorf("unexpected added node %q is also in removed for shard %q", node, shardName)
			}

			// pick a random source node from the remaining nodes
			var sourceNode string
			i := rand.Intn(len(nodes.Remaining))
			j := 0
			for n := range nodes.Remaining {
				if j == i {
					sourceNode = n
					break
				}
				j++
			}

			scalePlan.ShardReplicationScaleActions[shardName].AddNodes[node] = sourceNode

View on GitHub (pinned to 75aa4b6d11)

Solutions

1. Subtract the current replica set from the desired set before classifying nodes as added
2. Classify nodes present in both before and after as Remaining, not Added
3. Validate Added ∩ Remaining = ∅ before submitting the request
4. Re-query the scale plan so the server computes the diff itself

Example fix

// before
remaining: {node1}, added: {node1, node2} // node1 double-counted
// after
remaining: {node1}, added: {node2}

added := setDifference(desired, current) // only truly new nodes
remaining := intersect(desired, current)
// assert added ∩ remaining == ∅

plan, err := mgr.QueryReplicationScalePlan(req)
if err != nil && strings.Contains(err.Error(), "added node") && strings.Contains(err.Error(), "is also in remaining") {
    // recompute added set as desired minus current
}

When it happens

Trigger: Calling QueryReplicationScalePlan where the desired assignment includes an existing shard node in the Added set (not just Remaining), typically because the diff classified already-present nodes as new.

Common situations: Diff tooling comparing full node lists instead of set differences (not subtracting current replicas); applying the same scale plan twice with unrefreshed state.

Related errors

• unexpected removed node %q is also in remaining for shard %q
• unexpected removed node %q is also in added for shard %q
• unexpected added node %q is also in removed for shard %q
• replication operation with id %v is not in a state to be del
• cannot determine source node for shard %q: no remaining node

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/db93409d3f7995b4. Report an issue: GitHub.