{"record":{"id":"eebf652d8877ec2e","repo":"dgraph-io/dgraph","slug":"cannot-assign-tablet-for-pred-s-to-group-d","errorCode":null,"errorMessage":"cannot assign tablet for pred %s to group %d","messagePattern":"cannot assign tablet for pred (.+?) to group (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"worker/online_restore.go","lineNumber":335,"sourceCode":"\t\t//\n\t\t// Skipping ForceTablet is safe because the restore mapper decides which\n\t\t// data keys to restore based on the manifest's predicate set (predSet),\n\t\t// not on Zero's tablet assignments. The supporting predicate data will\n\t\t// still be restored to the correct Alpha's Badger store.\n\t\tif strings.HasSuffix(pred, hnsw.VecEntry) ||\n\t\t\tstrings.HasSuffix(pred, hnsw.VecKeyword) ||\n\t\t\tstrings.HasSuffix(pred, hnsw.VecDead) {\n\t\t\tcontinue\n\t\t}\n\n\t\t// Force the tablet to be moved to this group, even\n\t\t// if it's currently being served by another group.\n\t\ttablet, err := groups().ForceTablet(pred)\n\t\tif err != nil {\n\t\t\treturn errors.Wrapf(err, \"cannot create tablet for restored predicate %s\", pred)\n\t\t}\n\t\tif tablet.GetGroupId() != req.GroupId {\n\t\t\treturn errors.Errorf(\"cannot assign tablet for pred %s to group %d\", pred, req.GroupId)\n\t\t}\n\t}\n\n\tmapDir, err := os.MkdirTemp(x.WorkerConfig.TmpDir, \"restore-map\")\n\tx.Check(err)\n\tdefer func() {\n\t\tif err := os.RemoveAll(mapDir); err != nil {\n\t\t\tglog.Warningf(\"Error removing temp restore-map dir: %v\", err)\n\t\t}\n\t}()\n\tglog.Infof(\"Created temporary map directory: %s\\n\", mapDir)\n\n\t// Map the backup.\n\tmapRes, err := RunMapper(req, mapDir)\n\tif err != nil {\n\t\treturn errors.Wrapf(err, \"Failed to map the backup files\")\n\t}\n\tglog.Infof(\"Backup map phase is complete. Map result is: %+v\\n\", mapRes)","sourceCodeStart":317,"sourceCodeEnd":353,"githubUrl":"https://github.com/dgraph-io/dgraph/blob/759e242be62c91f8d084da06ad0c8d21256d9c07/worker/online_restore.go#L317-L353","documentation":"Returned by handleRestoreProposal (worker/online_restore.go:335) when ForceTablet succeeds but the returned tablet's GroupId is not req.GroupId — Zero placed the predicate's tablet on a different group than the one restoring the backup data. Since this Alpha only has the backup data for its group, the predicate cannot be restored correctly here and restore aborts.","triggerScenarios":"Zero moved/reassigned the tablet to another group between the ForceTablet call and the check (concurrent moveTablet or rebalance); a race with another restore proposal for the same predicate; Zero membership state diverging from the restore group's expectation.","commonSituations":"Running restores on multiple groups concurrently so each tries to claim the same predicates; tablet rebalancing triggered by a new Alpha joining mid-restore; ZooKeeper-free membership churn after Zero restart.","solutions":["Retry the restore when no tablet moves are in progress; ensure the tablet for the predicate ends on the restoring group.","Run restores sequentially per group and disable concurrent moveTablet/rebalancing operations during restore.","Verify Zero membership state (curl localhost:6080/state) shows the predicate assigned to req.GroupId before retrying.","If it persists, restart Alphas/Zero to clear membership churn, then restore again."],"exampleFix":"# before: concurrent restores + rebalancing race\n# after: move tablet to the right group and retry\ncurl localhost:6080/moveTablet?tablet=email&group=1   # pin pred to group 1\ncurl -X POST localhost:8080/restore -d '{\"groupId\": 1, \"location\": \"s3://bucket/backup\"}'","handlingStrategy":"retry","validationCode":"// Before restore, confirm predicate tablets map to req.GroupId\nresp, _ := http.Get(\"http://zero:6080/state\")\n// check tablets for backup predicates are assigned to the restoring group","typeGuard":null,"tryCatchPattern":"err := restore(ctx, req)\nif err != nil && strings.Contains(err.Error(), \"cannot assign tablet for pred\") {\n    // pin tablet to the expected group, wait, then retry\n    // curl zero:6080/moveTablet?tablet=<pred>&group=<reqGroupId>\n}","preventionTips":["Restore groups sequentially, not concurrently, to avoid tablet-claim races.","Disable tablet rebalancing/moveTablet operations during a restore.","Verify Zero tablet assignments via /state before each group's restore.","Restart Zero/Alpha to clear membership churn if assignments keep drifting."],"tags":["dgraph","restore","tablet","zero","race"],"backgroundTag":"tablet-assigned-to-wrong-group","analyzedSha":"759e242be62c91f8d084da06ad0c8d21256d9c07","analyzedAt":"2026-09-01T14:42:12.034Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}