Move all tablets from group %d before removing the last node

Error message

Move all tablets from group %d before removing the last node

What it means

Zero refuses to remove the last remaining node of a group while that group still serves tablets. Removing the final member would strand predicates with no server to serve them, so RemoveNode blocks the proposal via proposeAndWait never being reached. The operator must first move (or drop) all tablets out of the group.

Source

Thrown at dgraph/cmd/zero/zero.go:494

// RemoveNode removes the given node from the given group.
// It's the user's responsibility to ensure that node doesn't come back again
// before calling the api.
func (s *Server) RemoveNode(ctx context.Context, req *pb.RemoveNodeRequest) (*pb.Status, error) {
	if req.GroupId == 0 {
		return nil, s.Node.ProposePeerRemoval(ctx, req.NodeId)
	}
	zp := &pb.ZeroProposal{}
	zp.Member = &pb.Member{Id: req.NodeId, GroupId: req.GroupId, AmDead: true}
	if _, ok := s.state.Groups[req.GroupId]; !ok {
		return nil, errors.Errorf("No group with groupId %d found", req.GroupId)
	}
	if _, ok := s.state.Groups[req.GroupId].Members[req.NodeId]; !ok {
		return nil, errors.Errorf("No node with nodeId %d found in group %d", req.NodeId,
			req.GroupId)
	}
	if len(s.state.Groups[req.GroupId].Members) == 1 && len(s.state.Groups[req.GroupId].
		Tablets) > 0 {
		return nil, errors.Errorf("Move all tablets from group %d before removing the last node",
			req.GroupId)
	}
	if err := s.Node.proposeAndWait(ctx, zp); err != nil {
		return nil, err
	}

	return &pb.Status{}, nil
}

// Connect is used by Alpha nodes to connect the very first time with group zero.
func (s *Server) Connect(ctx context.Context,
	m *pb.Member) (resp *pb.ConnectionState, err error) {
	// Ensures that connect requests are always serialized
	s.connectLock.Lock()
	defer s.connectLock.Unlock()
	glog.Infof("Got connection request: %+v\n", m)
	defer glog.Infof("Connected: %+v\n", m)

View on GitHub (pinned to 759e242be6)

Solutions

Move all tablets out of the group (use /moveTablet on Zero or the MoveTablet API) to another healthy group, then retry RemoveNode.
If the data is no longer needed, drop the predicates/flag the tablets read-only and let them be deleted so group.Tablets becomes empty.
As a last resort, remove the whole group's data intentionally (export, then remove nodes) and let the group re-form.
If you actually meant to remove a node from a multi-member group, pass the correct NodeId of a non-last member.

Example fix

// before
zero.RemoveNode(ctx, &pb.RemoveNodeRequest{GroupId: 2, NodeId: 3}) // group 2 still serves tablets

// after
// 1. move tablets: curl 'http://localhost:6080/moveTablet?tablet=app.user&from=2&to=1'
// 2. once group 2 has no tablets:
zero.RemoveNode(ctx, &pb.RemoveNodeRequest{GroupId: 2, NodeId: 3})

Defensive patterns

Strategy: validation

Validate before calling

state := fetchZeroState("http://zero:6080/state")
g := state.Groups[groupId]
if g != nil && len(g.Members) == 1 && len(g.Tablets) > 0 {
    return fmt.Errorf("group %d still serves %d tablets; move them first", groupId, len(g.Tablets))
}

Try / catch

resp, err := zero.RemoveNode(ctx, req)
if err != nil && strings.Contains(err.Error(), "Move all tablets") {
    if merr := moveAllTablets(ctx, req.GroupId); merr != nil { return merr }
    return zero.RemoveNode(ctx, req)
}
return err

Prevention

Before removing any node, run /state and confirm the target group has zero tablets or more than one member.
Automate tablet draining (moveTablet) as a pre-step in decommission runbooks.
Script removals to check membership state instead of blind RPC calls.

When it happens

Trigger: Calling Zero's RemoveNode RPC (or `dgraph zero --remove` / MoveTablet flows) with req.NodeId being the only entry in s.state.Groups[groupId].Members while len(s.state.Groups[groupId].Tablets) > 0.

Common situations: Decommissioning a single-node alpha group; shrinking a cluster where a group has only one server but still owns predicates; scripting node removal without first rebalancing tablets.

Related errors

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/4a2b190c2d708c3d. Report an issue: GitHub.