dgraph-io/dgraph · error

cannot assign tablet for pred %s to group %d

Error message

cannot assign tablet for pred %s to group %d

What it means

Returned by handleRestoreProposal (worker/online_restore.go:335) when ForceTablet succeeds but the returned tablet's GroupId is not req.GroupId — Zero placed the predicate's tablet on a different group than the one restoring the backup data. Since this Alpha only has the backup data for its group, the predicate cannot be restored correctly here and restore aborts.

Source

Thrown at worker/online_restore.go:335

		//
		// Skipping ForceTablet is safe because the restore mapper decides which
		// data keys to restore based on the manifest's predicate set (predSet),
		// not on Zero's tablet assignments. The supporting predicate data will
		// still be restored to the correct Alpha's Badger store.
		if strings.HasSuffix(pred, hnsw.VecEntry) ||
			strings.HasSuffix(pred, hnsw.VecKeyword) ||
			strings.HasSuffix(pred, hnsw.VecDead) {
			continue
		}

		// Force the tablet to be moved to this group, even
		// if it's currently being served by another group.
		tablet, err := groups().ForceTablet(pred)
		if err != nil {
			return errors.Wrapf(err, "cannot create tablet for restored predicate %s", pred)
		}
		if tablet.GetGroupId() != req.GroupId {
			return errors.Errorf("cannot assign tablet for pred %s to group %d", pred, req.GroupId)
		}
	}

	mapDir, err := os.MkdirTemp(x.WorkerConfig.TmpDir, "restore-map")
	x.Check(err)
	defer func() {
		if err := os.RemoveAll(mapDir); err != nil {
			glog.Warningf("Error removing temp restore-map dir: %v", err)
		}
	}()
	glog.Infof("Created temporary map directory: %s\n", mapDir)

	// Map the backup.
	mapRes, err := RunMapper(req, mapDir)
	if err != nil {
		return errors.Wrapf(err, "Failed to map the backup files")
	}
	glog.Infof("Backup map phase is complete. Map result is: %+v\n", mapRes)

View on GitHub (pinned to 759e242be6)

Solutions

1. Retry the restore when no tablet moves are in progress; ensure the tablet for the predicate ends on the restoring group.
2. Run restores sequentially per group and disable concurrent moveTablet/rebalancing operations during restore.
3. Verify Zero membership state (curl localhost:6080/state) shows the predicate assigned to req.GroupId before retrying.
4. If it persists, restart Alphas/Zero to clear membership churn, then restore again.

Example fix

# before: concurrent restores + rebalancing race
# after: move tablet to the right group and retry
curl localhost:6080/moveTablet?tablet=email&group=1   # pin pred to group 1
curl -X POST localhost:8080/restore -d '{"groupId": 1, "location": "s3://bucket/backup"}'

// Before restore, confirm predicate tablets map to req.GroupId
resp, _ := http.Get("http://zero:6080/state")
// check tablets for backup predicates are assigned to the restoring group

err := restore(ctx, req)
if err != nil && strings.Contains(err.Error(), "cannot assign tablet for pred") {
    // pin tablet to the expected group, wait, then retry
    // curl zero:6080/moveTablet?tablet=<pred>&group=<reqGroupId>
}

When it happens

Trigger: Zero moved/reassigned the tablet to another group between the ForceTablet call and the check (concurrent moveTablet or rebalance); a race with another restore proposal for the same predicate; Zero membership state diverging from the restore group's expectation.

Common situations: Running restores on multiple groups concurrently so each tries to claim the same predicates; tablet rebalancing triggered by a new Alpha joining mid-restore; ZooKeeper-free membership churn after Zero restart.

Related errors

• cannot create tablet for restored predicate %s
• group: [%d] is not a known group
• namespace: %d. No tablet found for: %s
• namespace: %d. Tablet: [%s] is already being served by group
• Unable to move reserved predicate %s

AI-assisted analysis of dgraph-io/dgraph@759e242be6 (2026-09-01). Data as JSON: /api/errors/eebf652d8877ec2e. Report an issue: GitHub.