{"record":{"id":"7773c7d3678a9c02","repo":"vitessio/vitess","slug":"getkeyspace-s-failed-to-acquire-toporeadpool-w","errorCode":null,"errorMessage":"GetKeyspace(%s) failed to acquire topoReadPool: %w","messagePattern":"GetKeyspace\\((.+?)\\) failed to acquire topoReadPool: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"go/vt/vtadmin/cluster/cluster.go","lineNumber":1181,"sourceCode":"\tcpb := c.ToProto()\n\n\tfor _, g := range gates {\n\t\tg.Cluster = cpb\n\t}\n\n\treturn gates, nil\n}\n\n// GetKeyspace returns a single keyspace in the cluster.\nfunc (c *Cluster) GetKeyspace(ctx context.Context, name string) (*vtadminpb.Keyspace, error) {\n\tspan, ctx := trace.NewSpan(ctx, \"Cluster.GetKeyspace\")\n\tdefer span.Finish()\n\n\tAnnotateSpan(c, span)\n\tspan.Annotate(\"keyspace\", name)\n\n\tif err := c.topoReadPool.Acquire(ctx); err != nil {\n\t\treturn nil, fmt.Errorf(\"GetKeyspace(%s) failed to acquire topoReadPool: %w\", name, err)\n\t}\n\tdefer c.topoReadPool.Release()\n\n\tresp, err := c.Vtctld.GetKeyspace(ctx, &vtctldatapb.GetKeyspaceRequest{\n\t\tKeyspace: name,\n\t})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tshards, err := c.FindAllShardsInKeyspace(ctx, name, FindAllShardsInKeyspaceOptions{\n\t\tskipPool: true, // we already acquired before making the GetKeyspace call\n\t})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &vtadminpb.Keyspace{","sourceCodeStart":1163,"sourceCodeEnd":1199,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtadmin/cluster/cluster.go#L1163-L1199","documentation":"Cluster.GetKeyspace acquires a semaphore (topoReadPool) before making the Vtctld GetKeyspace RPC; Acquire fails if the context is cancelled/timed out while waiting for a slot. The wrap attributes the failure to the specific keyspace request. It signals pool exhaustion or request cancellation, not a keyspace problem.","triggerScenarios":"Calling cluster.GetKeyspace(ctx, name) when all topoReadPool slots are busy and ctx is cancelled before a slot frees, or ctx deadline exceeded during Acquire.","commonSituations":"Burst of concurrent GetKeyspace/GetKeyspaces calls from vtadmin web UI exceeding the configured topo read pool size; slow vtctld causing long-held slots; HTTP request timeouts cancelling ctx mid-wait.","solutions":["Check the wrapped cause: context.Canceled means the caller gave up; context.DeadlineExceeded means slots stayed busy too long","Increase the topo read pool size in vtadmin configuration","Reduce concurrent keyspace requests (rate-limit UI/API callers)","Investigate why vtctld GetKeyspace calls are slow enough to hold pool slots"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Go: skip the call if the context is already done\nif err := ctx.Err(); err != nil {\n\treturn fmt.Errorf(\"context done before GetKeyspace: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"ks, err := c.GetKeyspace(ctx, name)\nif err != nil {\n\tvar ctxErr error\n\tif errors.As(err, &ctxErr) && (errors.Is(ctxErr, context.DeadlineExceeded)) {\n\t\t// pool exhaustion; retry with backoff and a longer deadline\n\t}\n\treturn err\n}","preventionTips":["Size topoReadPool for peak concurrent keyspace reads","Use generous but bounded request deadlines","Rate-limit bulk keyspace API usage","Monitor pool acquisition latency"],"tags":["vtadmin","semaphore","concurrency","timeout"],"backgroundTag":"read-pool-exhausted","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}