vitessio/vitess · error

permissions differ on %v %v: %s: %v differs from: %s: %v

Error message

permissions differ on %v %v:
%s: %v
 differs from:
%s: %v

What it means

diffPermissions compares two lists of MySQL permission entries (users or db grants) from two tablets. When an entry with the same name/key exists on both sides but its serialized value differs, this error records that the permission definitions diverge. It is recorded via an ErrorRecorder as part of DiffPermissions, typically during tablet permission consistency checks (vtctldclient GetPermissions diff).

Source

Thrown at go/vt/mysqlctl/tmutils/permissions.go:192

		rpk, rval := right.Get(rightIndex)

		// extra value on the left side
		if lpk < rpk {
			er.RecordError(fmt.Errorf("%v has an extra %v %v", leftName, name, lpk))
			leftIndex++
			continue
		}

		// extra value on the right side
		if lpk > rpk {
			er.RecordError(fmt.Errorf("%v has an extra %v %v", rightName, name, rpk))
			rightIndex++
			continue
		}

		// same name, let's see content
		if lval != rval {
			er.RecordError(fmt.Errorf("permissions differ on %v %v:\n%s: %v\n differs from:\n%s: %v", name, lpk, leftName, lval, rightName, rval))
		}
		leftIndex++
		rightIndex++
	}
	for leftIndex < left.Len() {
		lpk, _ := left.Get(leftIndex)
		er.RecordError(fmt.Errorf("%v has an extra %v %v", leftName, name, lpk))
		leftIndex++
	}
	for rightIndex < right.Len() {
		rpk, _ := right.Get(rightIndex)
		er.RecordError(fmt.Errorf("%v has an extra %v %v", rightName, name, rpk))
		rightIndex++
	}
}

// DiffPermissions records the errors between two permission sets
func DiffPermissions(leftName string, left *tabletmanagerdatapb.Permissions, rightName string, right *tabletmanagerdatapb.Permissions, er concurrency.ErrorRecorder) {

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Run the missing GRANT statement(s) from the left side on the right tablet's MySQL (or vice versa) so both match
2. Re-sync permissions by copying grants from a known-good tablet (mysqldump mysql.user/mysql_db tables or SHOW GRANTS)
3. Check replication is healthy on the lagging tablet and let it catch up if drift was caused by a replicated grant
4. Re-provision the tablet from the primary's current state

Example fix

// before (drifted)
-- tablet A: GRANT SELECT, INSERT ON app.* TO 'reader'@'%'
-- tablet B: GRANT SELECT ON app.* TO 'reader'@'%'
// after
-- run on tablet B's MySQL:
GRANT SELECT, INSERT ON app.* TO 'reader'@'%';

leftPerms, err := getPermissions(ctx, leftTablet)
if err != nil { return err }
rightPerms, err := getPermissions(ctx, rightTablet)
if err != nil { return err }
if diffs := tmutils.DiffPermissionsToArray("left", leftPerms, "right", rightPerms); len(diffs) > 0 {
    for _, d := range diffs { log.Warn("permission diff", slog.String("detail", d)) }
}

er := concurrency.AllErrorRecorder{}
tmutils.DiffPermissions("left", leftPerms, "right", rightPerms, &er)
if er.HasErrors() {
    for _, err := range er.Errors {
        log.Warn("permission drift detected", slog.Any("error", err))
    }
    return er.Error()
}

When it happens

Trigger: Calling DiffPermissions (or DiffPermissionsToArray / the GetPermissions diff workflow) where two tablets' Permissions protos contain the same user or db grant name but different grant contents (e.g. different privileges on the same user).

Common situations: Grant was applied on one MySQL replica but not the other; manual GRANT/REVOKE run on one shard replica only; a replica restored from an older backup missing newer grants; ft-myisam or repl-grant drift after provisioning tablets from different snapshots.

Related errors

• schemas are different: %s: %v, %s: %v
• schemas are different: %s: %v differs from: %s: %v
• %v has an extra table named %v
• schemas differ on table %v: %s: %v differs from: %s: %v
• SHOW BINARY LOGS returned no rows

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/565803f7dd3800bf. Report an issue: GitHub.