dgraph-io/dgraph · error

while proposing tablet reassignment. Proposal: %+v

Error message

while proposing tablet reassignment. Proposal: %+v

What it means

This error wraps a failure from Zero's raft proposeAndWait call when moving a predicate (tablet) from one group to another. movePredicate proposes a MembershipChange/MoveTablet proposal to the Zero raft ring; if the proposal is not committed (lost leadership, shutdown, context cancel), the underlying error is wrapped with the full proposal for diagnosis.

Source

Thrown at dgraph/cmd/zero/tablet.go:225

	glog.Infof("Starting move: %+v", in)
	if _, err := wc.MovePredicate(ctx, in); err != nil {
		return errors.Wrapf(err, "while calling MovePredicate")
	}

	p := &pb.ZeroProposal{}
	p.Tablet = &pb.Tablet{
		GroupId:           dstGroup,
		Predicate:         predicate,
		OnDiskBytes:       tab.OnDiskBytes,
		UncompressedBytes: tab.UncompressedBytes,
		Force:             true,
		MoveTs:            in.TxnTs,
	}
	msg = fmt.Sprintf("Move at Alpha done. Now proposing: %+v", p)
	span.AddEvent(fmt.Sprintf("Zero proposal: %+v", p))
	glog.Info(msg)
	if err := s.Node.proposeAndWait(ctx, p); err != nil {
		return errors.Wrapf(err, "while proposing tablet reassignment. Proposal: %+v", p)
	}
	msg = fmt.Sprintf("Predicate move done for: [%v] from group %d to %d\n",
		predicate, srcGroup, dstGroup)
	span.AddEvent(msg)
	glog.Info(msg)

	// Now that the move has happened, we can delete the predicate from the source group. But before
	// doing that, we should ensure the source group understands that the predicate is now being
	// served by the destination group. For that, we pass in the expected checksum for the source
	// group. Only once the source group membership checksum matches, would the source group delete
	// the predicate. This ensures that it does not service any transaction after deletion of data.
	checksums := s.groupChecksums()
	in.ExpectedChecksum = checksums[in.SourceGid]
	in.DestGid = 0 // Indicates deletion of predicate in the source group.
	if _, err := wc.MovePredicate(ctx, in); err != nil {
		msg = fmt.Sprintf("While deleting predicate [%v] in group %d. Error: %v",
			in.Predicate, in.SourceGid, err)
		span.AddEvent(msg)

View on GitHub (pinned to 759e242be6)

Solutions

  1. Check Zero cluster health and ensure a leader exists (curl localhost:6080/state), then retry the move after quorum is restored.
  2. Retry the moveTablet request; rebalanceTablets runs periodically and will retry automatically.
  3. Verify the predicate is not already served by the destination group (the move may have partially applied).
  4. Inspect Zero logs for the underlying raft error (lost leadership, no quorum) preceding this wrap.

Example fix

// before
s.Node.proposeAndWait(ctx, p)
// after
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
if err := s.Node.proposeAndWait(ctx, p); err != nil {
    glog.Errorf("tablet move failed, will retry: %v", err)
    return err
}
Defensive patterns

Strategy: retry

Validate before calling

state := fetchZeroState("http://zero:6080/state")
if !state.HasZeroLeader() { return errors.New("no Zero leader; cannot move tablet") }

Try / catch

err := moveTablet(ctx, predicate, src, dst)
if err != nil && strings.Contains(err.Error(), "while proposing tablet reassignment") {
    // wait for quorum/leadership, then retry
    time.Sleep(10 * time.Second)
    return moveTablet(ctx, predicate, src, dst)
}

Prevention

When it happens

Trigger: Calling /moveTablet or rebalanceTablets when Zero loses leadership mid-proposal, the raft quorum is unavailable, or the context is canceled before the proposal is committed.

Common situations: Rebalancing tablets in a cluster with only one healthy Zero (no quorum); issuing moveTablet during a Zero restart; network partition between Zero nodes while a rebalance is running.

Related errors


AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/66d7cdff855c30e2. Report an issue: GitHub.