vitessio/vitess · error

cannot find serving, non-primary tablet in keyspace=%s: %w

Error message

cannot find serving, non-primary tablet in keyspace=%s: %w

What it means

After validation, VTAdmin searches the cluster's tablet inventory for a tablet that is in the serving graph, not PRIMARY, and in SERVING state within the requested keyspace. If FindTablet returns an error (no such tablet exists), the request fails with this wrapped error. VTEXplain must run the query against a replica/rdonly tablet, so none being available is fatal.

Source

Thrown at go/vt/vtadmin/api.go:2706

	if !api.authz.IsAuthorized(ctx, c.ID, rbac.VTExplainResource, rbac.GetAction) {
		return nil, nil
	}

	lockWaitStart := time.Now()

	api.vtexplainLock.Lock()
	defer api.vtexplainLock.Unlock()

	lockWaitTime := time.Since(lockWaitStart)
	log.Info(fmt.Sprintf("vtexplain lock wait time: %s", lockWaitTime))

	span.Annotate("vtexplain_lock_wait_time", lockWaitTime.String())

	tablet, err := c.FindTablet(ctx, func(t *vtadminpb.Tablet) bool {
		return t.Tablet.Keyspace == req.Keyspace && topo.IsInServingGraph(t.Tablet.Type) && t.Tablet.Type != topodatapb.TabletType_PRIMARY && t.State == vtadminpb.Tablet_SERVING
	})
	if err != nil {
		return nil, fmt.Errorf("cannot find serving, non-primary tablet in keyspace=%s: %w", req.Keyspace, err)
	}

	span.Annotate("tablet_alias", topoproto.TabletAliasString(tablet.Tablet.Alias))

	var (
		wg sync.WaitGroup
		er concurrency.AllErrorRecorder

		// Writes to these three variables are, in the strictest sense, unsafe.
		// However, there is one goroutine responsible for writing each of these
		// values (so, no concurrent writes), and reads are blocked on the call to
		// wg.Wait(), so we guarantee that all writes have finished before attempting
		// to read anything.
		srvVSchema string
		schema     string
		shardMap   string
	)

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Bring up at least one replica or rdonly tablet in the keyspace and ensure it reports SERVING (check with `vtctldclient GetTablets`).
2. Fix the tablet's state: verify mysqld is running, the tablet is started (`vtctldclient StartTablet`) and its health check passes so it enters the serving graph.
3. Confirm the keyspace name in the request matches an existing keyspace (typos yield the same 'not found' error).
4. Check the topo (etcd/zk) connectivity from vtadmin if tablets exist but aren't visible to it.
5. If using a shard-targeted setup, ensure the tablet's key range/shard covers the keyspace being queried.

Example fix

// before (all tablets down)
tablet, err := c.FindTablet(...) // errors: cannot find serving, non-primary tablet
// after: start a replica and wait for it to serve
vtctldclient StartTablet zone1-0000000101
# tablet.Type=REPLICA, State=SERVING -> FindTablet succeeds

// Check a serving non-primary tablet exists before calling
tablets, err := apiClient.GetTablets(ctx, &vtadminpb.GetTabletsRequest{Cluster: clusterID})
if err != nil { return err }
found := false
for _, t := range tablets.Tablets {
    if t.Tablet.Keyspace == ks && topo.IsInServingGraph(t.Tablet.Type) &&
        t.Tablet.Type != topodatapb.TabletType_PRIMARY && t.State == vtadminpb.Tablet_SERVING {
        found = true; break
    }
}
if !found { return fmt.Errorf("no serving replica/rdonly tablet in keyspace %s", ks) }

func hasServingNonPrimary(tablets []*vtadminpb.Tablet, ks string) bool {
    for _, t := range tablets {
        if t.Tablet.Keyspace == ks && topo.IsInServingGraph(t.Tablet.Type) &&
            t.Tablet.Type != topodatapb.TabletType_PRIMARY && t.State == vtadminpb.Tablet_SERVING {
            return true
        }
    }
    return false
}

tablet, err := api.VTEXplain(ctx, req)
if err != nil && strings.Contains(err.Error(), "cannot find serving, non-primary tablet") {
    // surface an operator-facing message: bring up a replica and retry
    return vterrors.Errorf(vtrpcpb.Code_FAILED_PRECONDITION, "keyspace %s has no serving replica tablets", req.Keyspace)
}

When it happens

Trigger: Calling VTEXplain for a keyspace that has zero serving replica/rdonly tablets — e.g. all replicas are down, tablets are in a non-serving type, tablet state is NOT_SERVING, or the keyspace has only a primary.

Common situations: Shard's replicas are down for maintenance; tablets not yet registered/serving after a fresh deploy; keyspace recently created with only a primary; vttablet processes crashed; tablets shut down by cloud autoscaler.

Related errors

• %w: keyspace %s
• not allowed: deny-all security-policy enforced
• not allowed: read-only security-policy enforced
• invalid choice for enum
• value must be either a float64 (interpreted as seconds) or a

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/a3a69463465a04c2. Report an issue: GitHub.