vitessio/vitess · warning

GetKeyspace(%s) failed to acquire topoReadPool: %w

Error message

GetKeyspace(%s) failed to acquire topoReadPool: %w

What it means

Cluster.GetKeyspace acquires a semaphore (topoReadPool) before making the Vtctld GetKeyspace RPC; Acquire fails if the context is cancelled/timed out while waiting for a slot. The wrap attributes the failure to the specific keyspace request. It signals pool exhaustion or request cancellation, not a keyspace problem.

Source

Thrown at go/vt/vtadmin/cluster/cluster.go:1181

	cpb := c.ToProto()

	for _, g := range gates {
		g.Cluster = cpb
	}

	return gates, nil
}

// GetKeyspace returns a single keyspace in the cluster.
func (c *Cluster) GetKeyspace(ctx context.Context, name string) (*vtadminpb.Keyspace, error) {
	span, ctx := trace.NewSpan(ctx, "Cluster.GetKeyspace")
	defer span.Finish()

	AnnotateSpan(c, span)
	span.Annotate("keyspace", name)

	if err := c.topoReadPool.Acquire(ctx); err != nil {
		return nil, fmt.Errorf("GetKeyspace(%s) failed to acquire topoReadPool: %w", name, err)
	}
	defer c.topoReadPool.Release()

	resp, err := c.Vtctld.GetKeyspace(ctx, &vtctldatapb.GetKeyspaceRequest{
		Keyspace: name,
	})
	if err != nil {
		return nil, err
	}

	shards, err := c.FindAllShardsInKeyspace(ctx, name, FindAllShardsInKeyspaceOptions{
		skipPool: true, // we already acquired before making the GetKeyspace call
	})
	if err != nil {
		return nil, err
	}

	return &vtadminpb.Keyspace{

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Check the wrapped cause: context.Canceled means the caller gave up; context.DeadlineExceeded means slots stayed busy too long
2. Increase the topo read pool size in vtadmin configuration
3. Reduce concurrent keyspace requests (rate-limit UI/API callers)
4. Investigate why vtctld GetKeyspace calls are slow enough to hold pool slots

// Go: skip the call if the context is already done
if err := ctx.Err(); err != nil {
	return fmt.Errorf("context done before GetKeyspace: %w", err)
}

ks, err := c.GetKeyspace(ctx, name)
if err != nil {
	var ctxErr error
	if errors.As(err, &ctxErr) && (errors.Is(ctxErr, context.DeadlineExceeded)) {
		// pool exhaustion; retry with backoff and a longer deadline
	}
	return err
}

When it happens

Trigger: Calling cluster.GetKeyspace(ctx, name) when all topoReadPool slots are busy and ctx is cancelled before a slot frees, or ctx deadline exceeded during Acquire.

Common situations: Burst of concurrent GetKeyspace/GetKeyspaces calls from vtadmin web UI exceeding the configured topo read pool size; slow vtctld causing long-held slots; HTTP request timeouts cancelling ctx mid-wait.

Related errors

• GetWorkflows(keyspace = %s, active_only = %v) failed to acqu
• GetBackups(%s/%s) failed to acquire backupReadPool: %w
• GetCellInfo(%s) failed to acquire topoReadPool: %w
• GetKeyspaces() failed to acquire topoReadPool: %w
• invalid choice for enum

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/7773c7d3678a9c02. Report an issue: GitHub.