{"record":{"id":"2a900b9dfb4997b4","repo":"vitessio/vitess","slug":"permissions-diffs-v","errorCode":null,"errorMessage":"permissions diffs: %v","messagePattern":"permissions diffs: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtctl/grpcvtctldserver/server.go","lineNumber":4956,"sourceCode":"\t\t\t\t})\n\t\t\t\tif err != nil {\n\t\t\t\t\treturn err\n\t\t\t\t}\n\n\t\t\t\tlog.Info(fmt.Sprintf(\"Diffing permissions between %s and %s\", topoproto.TabletAliasString(referenceAlias),\n\t\t\t\t\ttopoproto.TabletAliasString(alias)))\n\t\t\t\ter := &concurrency.AllErrorRecorder{}\n\t\t\t\ttmutils.DiffPermissions(topoproto.TabletAliasString(referenceAlias), referencePermissions,\n\t\t\t\t\ttopoproto.TabletAliasString(alias), presp.Permissions, er)\n\t\t\t\tif er.HasErrors() {\n\t\t\t\t\treturn er.Error()\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t}\n\tif err := eg.Wait(); err != nil {\n\t\treturn nil, fmt.Errorf(\"permissions diffs: %v\", err)\n\t}\n\n\treturn &vtctldatapb.ValidatePermissionsKeyspaceResponse{}, nil\n}\n\n// ValidateSchemaKeyspace is a part of the vtctlservicepb.VtctldServer interface.\n// It will diff the schema between the tablets in all shards -- or a subset if\n// any specific shards are specified -- within the keyspace.\nfunc (s *VtctldServer) ValidateSchemaKeyspace(ctx context.Context, req *vtctldatapb.ValidateSchemaKeyspaceRequest) (resp *vtctldatapb.ValidateSchemaKeyspaceResponse, err error) {\n\tspan, ctx := trace.NewSpan(ctx, \"VtctldServer.ValidateSchemaKeyspace\")\n\tdefer span.Finish()\n\n\tdefer panicHandler(&err)\n\n\tspan.Annotate(\"keyspace\", req.Keyspace)\n\tspan.Annotate(\"shards\", req.Shards)\n\tkeyspace := req.Keyspace\n","sourceCodeStart":4938,"sourceCodeEnd":4974,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtctl/grpcvtctldserver/server.go#L4938-L4974","documentation":"ValidatePermissionsKeyspace aggregates permission diffs across all tablets in a keyspace using an errgroup; when any tablet's permission comparison fails, the group returns the first error and the server wraps it as \"permissions diffs: %v\". It signals that tablet permission validation could not complete cleanly for at least one tablet.","triggerScenarios":"Calling ValidatePermissionsKeyspace (vtctldclient ValidatePermissionsKeyspace) when a tablet RPC fails (tablet down, permission diff found recorded, topoSrv error) inside the errgroup.","commonSituations":"A replica tablet is unreachable during a validation run; tablet ACL/permission tables differ between primary and replicas after a config change or version upgrade.","solutions":["Check all tablets in the keyspace are serving and reachable (vtctldclient GetTablets).","Inspect vttablet logs of the failing tablet for the underlying permission error.","Re-run after fixing the tablet; permissions diffs that are real must be reconciled manually on the tablet."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"aliases, err := ts.FindAllTabletAliasesInKeyspace(ctx, ks); if err == nil { for _, a := range aliases { if _, err := ts.GetTablet(ctx, a); err != nil { /* tablet unreachable — fix before validating */ } } }","typeGuard":null,"tryCatchPattern":"resp, err := client.ValidatePermissionsKeyspace(ctx, &vtctldatapb.ValidatePermissionsKeyspaceRequest{Keyspace: ks}); if err != nil { if strings.Contains(err.Error(), \"permissions diffs:\") { log.Warn(\"permission validation incomplete\", slog.Any(\"error\", err)); /* inspect failing tablet, fix, retry */ } return err }","preventionTips":["Keep all tablets in the keyspace serving before running validation","Reconcile ACL/permission changes on every tablet, not just the primary","Run validation outside maintenance windows when tablets are stable"],"tags":["vtctld","validation","topo","rpc"],"backgroundTag":"tablet-validation-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}