What causes "GetKeyspaces() failed to acquire topoReadPool: %w" in vitessio/vitess?

Inspect the wrapped cause to distinguish cancellation vs deadline-exceeded

GetKeyspaces() failed to acquire topoReadPool

Error message

GetKeyspaces() failed to acquire topoReadPool: %w

What it means

Cluster.GetKeyspaces acquires topoReadPool before calling Vtctld.GetKeyspaces; failure to acquire means ctx cancellation/deadline while waiting for a free slot. This variant returns the error directly to the caller (unlike error 866 which records it into an error recorder).

Solutions

Inspect the wrapped cause to distinguish cancellation vs deadline-exceeded
Increase topoReadPool size in vtadmin config
Stagger or rate-limit bulk keyspace/shard operations
Check vtctld/topo latency that keeps slots occupied

Defensive patterns

Strategy: retry

Validate before calling

// Go: ensure ctx still alive and caller deadline sane before the call
if ctx.Err() != nil {
	return ctx.Err()
}

Try / catch

resp, err := c.GetKeyspaces(ctx)
if err != nil && strings.Contains(err.Error(), "failed to acquire topoReadPool") {
	// exponential backoff retry; or enlarge pool
}

Prevention

Increase topoReadPool size in vtadmin config
Avoid concurrent getShardSets storms; serialize or batch
Set realistic deadlines based on topo latency
Alert on repeated pool-acquisition failures

When it happens

Trigger: Calling cluster.GetKeyspaces(ctx) (e.g. via getShardSets) with all topoReadPool slots held and ctx cancelled before a slot becomes available.

Common situations: Many concurrent vtadmin schema/keyspace operations saturating the topo read pool; slow topo under load; caller-side timeouts firing while requests queue on the semaphore.

Related errors

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/1a6fbad2067964aa. Report an issue: GitHub.

Appendix: source

Thrown at go/vt/vtadmin/cluster/cluster.go:1214

		return nil, err
	}

	return &vtadminpb.Keyspace{
		Cluster:  c.ToProto(),
		Keyspace: resp.Keyspace,
		Shards:   shards,
	}, nil
}

// GetKeyspaces returns all keyspaces, with their shard maps, in the cluster.
func (c *Cluster) GetKeyspaces(ctx context.Context) ([]*vtadminpb.Keyspace, error) {
	span, ctx := trace.NewSpan(ctx, "Cluster.GetKeyspaces")
	defer span.Finish()

	AnnotateSpan(c, span)

	if err := c.topoReadPool.Acquire(ctx); err != nil {
		return nil, fmt.Errorf("GetKeyspaces() failed to acquire topoReadPool: %w", err)
	}

	resp, err := c.Vtctld.GetKeyspaces(ctx, &vtctldatapb.GetKeyspacesRequest{})
	c.topoReadPool.Release()

	if err != nil {
		return nil, err
	}

	var (
		m         sync.Mutex
		wg        sync.WaitGroup
		rec       concurrency.AllErrorRecorder
		keyspaces = make([]*vtadminpb.Keyspace, len(resp.Keyspaces))
	)

	for i, ks := range resp.Keyspaces {
		wg.Add(1)

View on GitHub (pinned to 01a25a7d17)