{"record":{"id":"565803f7dd3800bf","repo":"vitessio/vitess","slug":"permissions-differ-on-v-v-s-v-differs-from","errorCode":null,"errorMessage":"permissions differ on %v %v:\n%s: %v\n differs from:\n%s: %v","messagePattern":"permissions differ on (.+?) (.+?):\n(.+?): (.+?)\n differs from:\n(.+?): (.+?)","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/mysqlctl/tmutils/permissions.go","lineNumber":192,"sourceCode":"\t\trpk, rval := right.Get(rightIndex)\n\n\t\t// extra value on the left side\n\t\tif lpk < rpk {\n\t\t\ter.RecordError(fmt.Errorf(\"%v has an extra %v %v\", leftName, name, lpk))\n\t\t\tleftIndex++\n\t\t\tcontinue\n\t\t}\n\n\t\t// extra value on the right side\n\t\tif lpk > rpk {\n\t\t\ter.RecordError(fmt.Errorf(\"%v has an extra %v %v\", rightName, name, rpk))\n\t\t\trightIndex++\n\t\t\tcontinue\n\t\t}\n\n\t\t// same name, let's see content\n\t\tif lval != rval {\n\t\t\ter.RecordError(fmt.Errorf(\"permissions differ on %v %v:\\n%s: %v\\n differs from:\\n%s: %v\", name, lpk, leftName, lval, rightName, rval))\n\t\t}\n\t\tleftIndex++\n\t\trightIndex++\n\t}\n\tfor leftIndex < left.Len() {\n\t\tlpk, _ := left.Get(leftIndex)\n\t\ter.RecordError(fmt.Errorf(\"%v has an extra %v %v\", leftName, name, lpk))\n\t\tleftIndex++\n\t}\n\tfor rightIndex < right.Len() {\n\t\trpk, _ := right.Get(rightIndex)\n\t\ter.RecordError(fmt.Errorf(\"%v has an extra %v %v\", rightName, name, rpk))\n\t\trightIndex++\n\t}\n}\n\n// DiffPermissions records the errors between two permission sets\nfunc DiffPermissions(leftName string, left *tabletmanagerdatapb.Permissions, rightName string, right *tabletmanagerdatapb.Permissions, er concurrency.ErrorRecorder) {","sourceCodeStart":174,"sourceCodeEnd":210,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/mysqlctl/tmutils/permissions.go#L174-L210","documentation":"diffPermissions compares two lists of MySQL permission entries (users or db grants) from two tablets. When an entry with the same name/key exists on both sides but its serialized value differs, this error records that the permission definitions diverge. It is recorded via an ErrorRecorder as part of DiffPermissions, typically during tablet permission consistency checks (vtctldclient GetPermissions diff).","triggerScenarios":"Calling DiffPermissions (or DiffPermissionsToArray / the GetPermissions diff workflow) where two tablets' Permissions protos contain the same user or db grant name but different grant contents (e.g. different privileges on the same user).","commonSituations":"Grant was applied on one MySQL replica but not the other; manual GRANT/REVOKE run on one shard replica only; a replica restored from an older backup missing newer grants; ft-myisam or repl-grant drift after provisioning tablets from different snapshots.","solutions":["Run the missing GRANT statement(s) from the left side on the right tablet's MySQL (or vice versa) so both match","Re-sync permissions by copying grants from a known-good tablet (mysqldump mysql.user/mysql_db tables or SHOW GRANTS)","Check replication is healthy on the lagging tablet and let it catch up if drift was caused by a replicated grant","Re-provision the tablet from the primary's current state"],"exampleFix":"// before (drifted)\n-- tablet A: GRANT SELECT, INSERT ON app.* TO 'reader'@'%'\n-- tablet B: GRANT SELECT ON app.* TO 'reader'@'%'\n// after\n-- run on tablet B's MySQL:\nGRANT SELECT, INSERT ON app.* TO 'reader'@'%';","handlingStrategy":"validation","validationCode":"leftPerms, err := getPermissions(ctx, leftTablet)\nif err != nil { return err }\nrightPerms, err := getPermissions(ctx, rightTablet)\nif err != nil { return err }\nif diffs := tmutils.DiffPermissionsToArray(\"left\", leftPerms, \"right\", rightPerms); len(diffs) > 0 {\n    for _, d := range diffs { log.Warn(\"permission diff\", slog.String(\"detail\", d)) }\n}","typeGuard":null,"tryCatchPattern":"er := concurrency.AllErrorRecorder{}\ntmutils.DiffPermissions(\"left\", leftPerms, \"right\", rightPerms, &er)\nif er.HasErrors() {\n    for _, err := range er.Errors {\n        log.Warn(\"permission drift detected\", slog.Any(\"error\", err))\n    }\n    return er.Error()\n}","preventionTips":["Apply grants through managed tooling (vtctl RebuildKeyspace / IaC) instead of manual GRANT statements on individual tablets","Run periodic permission diffs across all tablets in a keyspace","Keep grants identical at provision time via a shared bootstrap script","Watch replication health so replicated grant changes reach every tablet"],"tags":["mysql","permissions","consistency-check","vitess"],"backgroundTag":"permission-drift-between-tablets","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}