{"record":{"id":"aafa8bd9def7cdbf","repo":"vitessio/vitess","slug":"refreshtabletreplicationsource-v-failed-to-acqui","errorCode":null,"errorMessage":"RefreshTabletReplicationSource(%v) failed to acquire topoRWPool: %w","messagePattern":"RefreshTabletReplicationSource\\((.+?)\\) failed to acquire topoRWPool: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtadmin/cluster/cluster.go","lineNumber":2144,"sourceCode":"\tdefer c.topoReadPool.Release()\n\n\t_, err := c.Vtctld.RefreshState(ctx, &vtctldatapb.RefreshStateRequest{\n\t\tTabletAlias: tablet.Tablet.Alias,\n\t})\n\treturn err\n}\n\n// RefreshTabletReplicationSource performs a `CHANGE REPLICATION SOURCE TO` on\n// a tablet to replicate from the current primary in the shard.\nfunc (c *Cluster) RefreshTabletReplicationSource(ctx context.Context, tablet *vtadminpb.Tablet) (*vtadminpb.RefreshTabletReplicationSourceResponse, error) {\n\tspan, ctx := trace.NewSpan(ctx, \"Cluster.RefreshTabletReplicationSource\")\n\tdefer span.Finish()\n\n\tAnnotateSpan(c, span)\n\tspan.Annotate(\"tablet_alias\", topoproto.TabletAliasString(tablet.Tablet.Alias))\n\n\tif err := c.topoRWPool.Acquire(ctx); err != nil {\n\t\treturn nil, fmt.Errorf(\"RefreshTabletReplicationSource(%v) failed to acquire topoRWPool: %w\", topoproto.TabletAliasString(tablet.Tablet.Alias), err)\n\t}\n\tdefer c.topoRWPool.Release()\n\n\tresp, err := c.Vtctld.ReparentTablet(ctx, &vtctldatapb.ReparentTabletRequest{Tablet: tablet.Tablet.Alias})\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\treturn &vtadminpb.RefreshTabletReplicationSourceResponse{\n\t\tKeyspace: resp.Keyspace,\n\t\tShard:    resp.Shard,\n\t\tPrimary:  resp.Primary,\n\t\tCluster:  c.ToProto(),\n\t}, nil\n}\n\n// ReloadSchemas reloads schemas in one or more keyspaces, shards, or tablets\n// in the cluster, depending on the request parameters.","sourceCodeStart":2126,"sourceCodeEnd":2162,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtadmin/cluster/cluster.go#L2126-L2162","documentation":"Cluster.RefreshTabletReplicationSource (ReparentTablet) acquires topoRWPool before issuing a write to the topo. Acquire failure — cancelled context or pool exhaustion — is wrapped with the tablet alias and returned to the caller.","triggerScenarios":"Calling RefreshTabletReplicationSource while other topo writes (failovers, ReparentTablet, external reparent promotions) hold the limited topoRWPool slots, or with a ctx already cancelled.","commonSituations":"Repointing tablets during a failover storm when several concurrent reparent operations run; scripted tablet re-parenting loops; long-running EmergencyFailoverShard/PlannedFailoverShard holding the RW pool.","solutions":["Retry after concurrent topo write operations complete, with a fresh context","Serialize ReparentTablet calls client-side","Increase topo_rw_pool_size in the vtadmin cluster config","Verify no stuck operations are holding pool slots (check in-flight failovers)"],"exampleFix":"// before\nctx, cancel := context.WithTimeout(parentCtx, 500*time.Millisecond)\n_, err := c.RefreshTabletReplicationSource(ctx, tablet)\n// after\nctx, cancel := context.WithTimeout(parentCtx, 30*time.Second)\n_, err := c.RefreshTabletReplicationSource(ctx, tablet)","handlingStrategy":"retry","validationCode":"if ctx.Err() != nil {\n    return ctx.Err()\n}","typeGuard":null,"tryCatchPattern":"_, err := c.RefreshTabletReplicationSource(ctx, tablet)\nif err != nil && strings.Contains(err.Error(), \"failed to acquire topoRWPool\") {\n    // wait for in-flight reparents, then retry with fresh context\n}","preventionTips":["Avoid issuing ReparentTablet during concurrent failover operations","Serialize topo write operations in automation scripts","Keep topo_rw_pool_size aligned with concurrent write workflows"],"tags":["vtadmin","concurrency","topo-write","context-cancelled"],"backgroundTag":"semaphore-acquisition-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}