What causes "GetKeyspaces() failed to acquire topoReadPool: %w" in vitessio/vitess?

Inspect the wrapped cause to distinguish cancellation vs deadline-exceeded

GetKeyspaces() failed to acquire topoReadPool: %w

Error message

GetKeyspaces() failed to acquire topoReadPool: %w

What it means

Cluster.GetKeyspaces acquires topoReadPool before calling Vtctld.GetKeyspaces; failure to acquire means ctx cancellation/deadline while waiting for a free slot. This variant returns the error directly to the caller (unlike error 866 which records it into an error recorder).

Source

Thrown at go/vt/vtadmin/cluster/cluster.go:1214

		return nil, err
	}

	return &vtadminpb.Keyspace{
		Cluster:  c.ToProto(),
		Keyspace: resp.Keyspace,
		Shards:   shards,
	}, nil
}

// GetKeyspaces returns all keyspaces, with their shard maps, in the cluster.
func (c *Cluster) GetKeyspaces(ctx context.Context) ([]*vtadminpb.Keyspace, error) {
	span, ctx := trace.NewSpan(ctx, "Cluster.GetKeyspaces")
	defer span.Finish()

	AnnotateSpan(c, span)

	if err := c.topoReadPool.Acquire(ctx); err != nil {
		return nil, fmt.Errorf("GetKeyspaces() failed to acquire topoReadPool: %w", err)
	}

	resp, err := c.Vtctld.GetKeyspaces(ctx, &vtctldatapb.GetKeyspacesRequest{})
	c.topoReadPool.Release()

	if err != nil {
		return nil, err
	}

	var (
		m         sync.Mutex
		wg        sync.WaitGroup
		rec       concurrency.AllErrorRecorder
		keyspaces = make([]*vtadminpb.Keyspace, len(resp.Keyspaces))
	)

	for i, ks := range resp.Keyspaces {
		wg.Add(1)

View on GitHub (pinned to 01a25a7d17)

Solutions

Inspect the wrapped cause to distinguish cancellation vs deadline-exceeded
Increase topoReadPool size in vtadmin config
Stagger or rate-limit bulk keyspace/shard operations
Check vtctld/topo latency that keeps slots occupied

Defensive patterns

Strategy: retry

Validate before calling

// Go: ensure ctx still alive and caller deadline sane before the call
if ctx.Err() != nil {
	return ctx.Err()
}

Try / catch

resp, err := c.GetKeyspaces(ctx)
if err != nil && strings.Contains(err.Error(), "failed to acquire topoReadPool") {
	// exponential backoff retry; or enlarge pool
}

Prevention

Increase topoReadPool size in vtadmin config
Avoid concurrent getShardSets storms; serialize or batch
Set realistic deadlines based on topo latency
Alert on repeated pool-acquisition failures

When it happens

Trigger: Calling cluster.GetKeyspaces(ctx) (e.g. via getShardSets) with all topoReadPool slots held and ctx cancelled before a slot becomes available.

Common situations: Many concurrent vtadmin schema/keyspace operations saturating the topo read pool; slow topo under load; caller-side timeouts firing while requests queue on the semaphore.

Related errors

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/1a6fbad2067964aa. Report an issue: GitHub.