vitessio/vitess · error

GetKeyspaces failed: %v

Error message

GetKeyspaces failed: %v

What it means

After determining the cells, RebuildSrvVSchema lists all keyspaces in the topology to build per-keyspace SrvVSchema entries. This error wraps a failure from GetKeyspaces, meaning the global keyspace listing could not be read and the rebuild aborts before writing anything.

Source

Thrown at go/vt/topo/srv_vschema.go:159

	return conn.Delete(ctx, nodePath, nil)
}

// RebuildSrvVSchema rebuilds the SrvVSchema for the provided cell list
// (or all cells if cell list is empty).
func (ts *Server) RebuildSrvVSchema(ctx context.Context, cells []string) error {
	// get the actual list of cells
	if len(cells) == 0 {
		var err error
		cells, err = ts.GetKnownCells(ctx)
		if err != nil {
			return fmt.Errorf("GetKnownCells failed: %v", err)
		}
	}

	// get the keyspaces
	keyspaces, err := ts.GetKeyspaces(ctx)
	if err != nil {
		return fmt.Errorf("GetKeyspaces failed: %v", err)
	}

	// build the SrvVSchema in parallel, protected by mu
	wg := sync.WaitGroup{}
	mu := sync.Mutex{}
	var finalErr error
	srvVSchema := &vschemapb.SrvVSchema{
		Keyspaces: map[string]*vschemapb.Keyspace{},
	}
	for _, keyspace := range keyspaces {
		wg.Add(1)
		go func(keyspace string) {
			defer wg.Done()

			ksvs, err := ts.GetVSchema(ctx, keyspace)
			if IsErrType(err, NoNode) {
				err = nil
				ksvs = &KeyspaceVSchemaInfo{

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Verify topo health and retry; GetKeyspaces is a read and safe to retry after a transient failure.
2. Check vtctld logs for the wrapped inner error (e.g. 'context deadline exceeded' vs 'permission denied') and address that root cause.
3. If a keyspace was deleted concurrently, re-run the rebuild after the delete settles — stale listings cause spurious failures.
4. Ensure the calling context has a generous timeout; large clusters can exceed short deadlines when listing keyspaces.

Example fix

// before
cctx, cancel := context.WithTimeout(ctx, 100*time.Millisecond)
err := ts.RebuildSrvVSchema(cctx, nil) // GetKeyspaces deadline exceeded
// after
cctx, cancel := context.WithTimeout(ctx, 30*time.Second)
err := ts.RebuildSrvVSchema(cctx, nil)

if _, err := ts.GetKeyspaces(ctx); err != nil {
    return fmt.Errorf("cannot list keyspaces, aborting: %w", err)
}

if err := ts.RebuildSrvVSchema(ctx, cells); err != nil {
    if strings.Contains(err.Error(), "GetKeyspaces failed") && errors.Is(ctx.Err(), nil) {
        return retryWithBackoff(ctx, func() error { return ts.RebuildSrvVSchema(ctx, cells) })
    }
    return err
}

When it happens

Trigger: Calling RebuildSrvVSchema (directly or via ApplyRoutingRules / InitTabletMap / vschema watcher) when the topo call to list keyspaces fails — topo backend down, context timeout, permission error, or corrupt keyspace directory.

Common situations: etcd compaction or session expiry during heavy vtctld activity; transient network blip between vtctld and topo; keyspace deleted concurrently by another process; RBAC/acl misconfiguration on the topo backend.

Related errors

• GetKnownCells failed: %v
• SrvVSchema has no entry for keyspace %v
• GetRoutingRules failed: %v
• GetShardRoutingRules failed: %v
• GetKeyspaceRoutingRules failed: %v

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/568722918a72aa06. Report an issue: GitHub.