{"record":{"id":"a3a69463465a04c2","repo":"vitessio/vitess","slug":"cannot-find-serving-non-primary-tablet-in-keyspac","errorCode":null,"errorMessage":"cannot find serving, non-primary tablet in keyspace=%s: %w","messagePattern":"cannot find serving, non-primary tablet in keyspace=(.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtadmin/api.go","lineNumber":2706,"sourceCode":"\tif !api.authz.IsAuthorized(ctx, c.ID, rbac.VTExplainResource, rbac.GetAction) {\n\t\treturn nil, nil\n\t}\n\n\tlockWaitStart := time.Now()\n\n\tapi.vtexplainLock.Lock()\n\tdefer api.vtexplainLock.Unlock()\n\n\tlockWaitTime := time.Since(lockWaitStart)\n\tlog.Info(fmt.Sprintf(\"vtexplain lock wait time: %s\", lockWaitTime))\n\n\tspan.Annotate(\"vtexplain_lock_wait_time\", lockWaitTime.String())\n\n\ttablet, err := c.FindTablet(ctx, func(t *vtadminpb.Tablet) bool {\n\t\treturn t.Tablet.Keyspace == req.Keyspace && topo.IsInServingGraph(t.Tablet.Type) && t.Tablet.Type != topodatapb.TabletType_PRIMARY && t.State == vtadminpb.Tablet_SERVING\n\t})\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"cannot find serving, non-primary tablet in keyspace=%s: %w\", req.Keyspace, err)\n\t}\n\n\tspan.Annotate(\"tablet_alias\", topoproto.TabletAliasString(tablet.Tablet.Alias))\n\n\tvar (\n\t\twg sync.WaitGroup\n\t\ter concurrency.AllErrorRecorder\n\n\t\t// Writes to these three variables are, in the strictest sense, unsafe.\n\t\t// However, there is one goroutine responsible for writing each of these\n\t\t// values (so, no concurrent writes), and reads are blocked on the call to\n\t\t// wg.Wait(), so we guarantee that all writes have finished before attempting\n\t\t// to read anything.\n\t\tsrvVSchema string\n\t\tschema     string\n\t\tshardMap   string\n\t)\n","sourceCodeStart":2688,"sourceCodeEnd":2724,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtadmin/api.go#L2688-L2724","documentation":"After validation, VTAdmin searches the cluster's tablet inventory for a tablet that is in the serving graph, not PRIMARY, and in SERVING state within the requested keyspace. If FindTablet returns an error (no such tablet exists), the request fails with this wrapped error. VTEXplain must run the query against a replica/rdonly tablet, so none being available is fatal.","triggerScenarios":"Calling VTEXplain for a keyspace that has zero serving replica/rdonly tablets — e.g. all replicas are down, tablets are in a non-serving type, tablet state is NOT_SERVING, or the keyspace has only a primary.","commonSituations":"Shard's replicas are down for maintenance; tablets not yet registered/serving after a fresh deploy; keyspace recently created with only a primary; vttablet processes crashed; tablets shut down by cloud autoscaler.","solutions":["Bring up at least one replica or rdonly tablet in the keyspace and ensure it reports SERVING (check with `vtctldclient GetTablets`).","Fix the tablet's state: verify mysqld is running, the tablet is started (`vtctldclient StartTablet`) and its health check passes so it enters the serving graph.","Confirm the keyspace name in the request matches an existing keyspace (typos yield the same 'not found' error).","Check the topo (etcd/zk) connectivity from vtadmin if tablets exist but aren't visible to it.","If using a shard-targeted setup, ensure the tablet's key range/shard covers the keyspace being queried."],"exampleFix":"// before (all tablets down)\ntablet, err := c.FindTablet(...) // errors: cannot find serving, non-primary tablet\n// after: start a replica and wait for it to serve\nvtctldclient StartTablet zone1-0000000101\n# tablet.Type=REPLICA, State=SERVING -> FindTablet succeeds","handlingStrategy":"validation","validationCode":"// Check a serving non-primary tablet exists before calling\ntablets, err := apiClient.GetTablets(ctx, &vtadminpb.GetTabletsRequest{Cluster: clusterID})\nif err != nil { return err }\nfound := false\nfor _, t := range tablets.Tablets {\n    if t.Tablet.Keyspace == ks && topo.IsInServingGraph(t.Tablet.Type) &&\n        t.Tablet.Type != topodatapb.TabletType_PRIMARY && t.State == vtadminpb.Tablet_SERVING {\n        found = true; break\n    }\n}\nif !found { return fmt.Errorf(\"no serving replica/rdonly tablet in keyspace %s\", ks) }","typeGuard":"func hasServingNonPrimary(tablets []*vtadminpb.Tablet, ks string) bool {\n    for _, t := range tablets {\n        if t.Tablet.Keyspace == ks && topo.IsInServingGraph(t.Tablet.Type) &&\n            t.Tablet.Type != topodatapb.TabletType_PRIMARY && t.State == vtadminpb.Tablet_SERVING {\n            return true\n        }\n    }\n    return false\n}","tryCatchPattern":"tablet, err := api.VTEXplain(ctx, req)\nif err != nil && strings.Contains(err.Error(), \"cannot find serving, non-primary tablet\") {\n    // surface an operator-facing message: bring up a replica and retry\n    return vterrors.Errorf(vtrpcpb.Code_FAILED_PRECONDITION, \"keyspace %s has no serving replica tablets\", req.Keyspace)\n}","preventionTips":["Monitor replica/rdonly tablet health per keyspace and alert when zero serving non-primary tablets remain.","Run vtexplain against keyspaces known to have healthy replicas.","After maintenance windows, verify tablets re-enter the serving graph before running vtexplain."],"tags":["vtadmin","vtexplain","topology","tablet-availability"],"backgroundTag":"no-serving-tablet-found","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}