{"record":{"id":"dc9bbcd293562785","repo":"vitessio/vitess","slug":"primaryposition-s-failed-w","errorCode":null,"errorMessage":"PrimaryPosition(%s) failed: %w","messagePattern":"PrimaryPosition\\((.+?)\\) failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtctl/grpcvtctldserver/server.go","lineNumber":4212,"sourceCode":"\n\t\t\t\tspan.Annotate(\"tablet_alias\", alias)\n\n\t\t\t\tctx, cancel := context.WithTimeout(ctx, topo.RemoteOperationTimeout)\n\t\t\t\tdefer cancel()\n\n\t\t\t\tvar status *replicationdatapb.Status\n\n\t\t\t\tpos, err := s.tmc.PrimaryPosition(ctx, tablet)\n\t\t\t\tif err != nil {\n\t\t\t\t\tswitch ctx.Err() {\n\t\t\t\t\tcase context.Canceled:\n\t\t\t\t\t\tlog.Warn(fmt.Sprintf(\"context canceled before obtaining primary position from %s: %s\", alias, err))\n\t\t\t\t\tcase context.DeadlineExceeded:\n\t\t\t\t\t\tlog.Warn(fmt.Sprintf(\"context deadline exceeded before obtaining primary position from %s: %s\", alias, err))\n\t\t\t\t\tdefault:\n\t\t\t\t\t\t// The RPC was not timed out or canceled. We treat this\n\t\t\t\t\t\t// as a fatal error for the overall request.\n\t\t\t\t\t\trec.RecordError(fmt.Errorf(\"PrimaryPosition(%s) failed: %w\", alias, err))\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t} else {\n\t\t\t\t\t// No error, record a valid status for this tablet.\n\t\t\t\t\tstatus = &replicationdatapb.Status{\n\t\t\t\t\t\tPosition: pos,\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\tm.Lock()\n\t\t\t\tdefer m.Unlock()\n\n\t\t\t\tresults[alias] = status\n\t\t\t\ttabletMap[alias] = tablet\n\t\t\t}(ctx, alias, tabletInfo.Tablet)\n\t\tcase tabletInfo.IsReplicaType():\n\t\t\twg.Add(1)\n","sourceCodeStart":4194,"sourceCodeEnd":4230,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtctl/grpcvtctldserver/server.go#L4194-L4230","documentation":"Aggregated error recorded when fetching the primary's replication position (PrimaryPosition RPC) fails for reasons other than context cancellation or deadline exceeded. Such a non-timeout failure is treated as fatal for the whole request and recorded via errgroup.","triggerScenarios":"A get-shard-replication-positions style request where the RPC to the primary tablet's tabletmanager (PrimaryPosition) fails with a connection error, tablet unavailable, or gRPC error — anything other than Canceled/DeadlineExceeded.","commonSituations":"Primary tablet is down or restarting during status gathering; tabletmanager port blocked by firewall/network partition; querying during a reparent when the old primary is unreachable.","solutions":["Check the primary tablet's health and that vttablet/tabletmanager is running (`vtctldclient GetTablet <alias>`)","Verify network connectivity between vtctld and the tablet's gRPC port","Retry the operation once the tablet is back or the reparent completes","If the primary alias is stale, repair the shard with a reparent"],"exampleFix":"// before\ndefault:\n    // The RPC was not timed out or canceled.\n    rec.RecordError(fmt.Errorf(\"PrimaryPosition(%s) failed: %w\", alias, err))\n    return\n// after\ndefault:\n    rec.RecordError(fmt.Errorf(\"PrimaryPosition(%s) failed: %w\", alias, err))\n    return\n// (caller side) treat partially available statuses as degraded rather than failing:\nstatuses := rec.GetStatuses()\nif len(statuses) == 0 { return nil, rec.GetError() }","handlingStrategy":"retry","validationCode":"// verify primary tablet is serving before gathering positions\n_, err := tmClient.Ping(ctx, primaryAlias)\nif err != nil { /* primary unreachable; skip or wait */ }","typeGuard":null,"tryCatchPattern":"try {\n  positions = await client.getShardReplicationPositions(keyspace, shard)\n} catch (e) {\n  if (String(e).includes('PrimaryPosition')) {\n    checkPrimaryHealth(); await backoffRetry(2)\n  } else { throw e }\n}","preventionTips":["Keep vttablet tabletmanager gRPC ports reachable from vtctld","Avoid running status gathering during reparents","Monitor primary tablet uptime and restart events","Check firewall/security-group rules between vtctld and tablets"],"tags":["replication","tabletmanager","grpc","primary"],"backgroundTag":"tablet-rpc-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}