{"record":{"id":"e4d56b5b846337a6","repo":"vitessio/vitess","slug":"failed-to-read-primary-instance-v-w","errorCode":null,"errorMessage":"failed to read primary instance %v: %w","messagePattern":"failed to read primary instance (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/vtorc/inst/instance_dao.go","lineNumber":445,"sourceCode":"\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to read shard primary for %s/%s: %w\", tablet.Keyspace, tablet.Shard, err)\n\t}\n\n\t// Check if the current tablet is the primary. If it is, then we don't need to\n\t// run errant GTID detection on it.\n\tif topoproto.TabletAliasEqual(primaryAlias, instance.InstanceAlias) {\n\t\t// A primary cannot have errant GTIDs relative to itself; clear any\n\t\t// value left over from before this tablet was promoted.\n\t\tinstance.GtidErrant = \"\"\n\t\tcurrentErrantGTIDCount.Reset(tabletAliasString)\n\t\treturn nil\n\t}\n\n\tvar primaryInstance *Instance\n\tif primaryAlias != nil {\n\t\tprimaryInstance, _, err = ReadInstance(primaryAlias)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to read primary instance %v: %w\", topoproto.TabletAliasString(primaryAlias), err)\n\t\t}\n\t}\n\n\t// Only run errant GTID detection if we are sure that the data read of the current primary\n\t// is up-to-date enough to reflect that it has been promoted. This is needed to prevent\n\t// flagging incorrect errant GTIDs. If we were to use old data, we could have some GTIDs\n\t// accepted by the old primary (this tablet) that don't show in the new primary's set.\n\tif primaryInstance != nil && primaryInstance.SourceHost == \"\" {\n\t\t// If the instance has no replication source and no primary GTID set yet, or if the instance's replication\n\t\t// source is not the primary, use the shard primary's executed GTID set for comparison.\n\t\tif (instance.SourceHost == \"\" && instance.primaryExecutedGtidSet == \"\") || !sourceIsPrimary(instance, primaryInstance) {\n\t\t\tinstance.primaryExecutedGtidSet = primaryInstance.ExecutedGtidSet\n\t\t}\n\t}\n\n\tvar errantGtidCount int64\n\tif instance.ExecutedGtidSet != \"\" && instance.primaryExecutedGtidSet != \"\" {\n\t\t// Compare primary & replica GTID sets, but ignore the sets that present the primary's UUID.","sourceCodeStart":427,"sourceCodeEnd":463,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/vtorc/inst/instance_dao.go#L427-L463","documentation":"After resolving the shard primary alias, vtorc reads that primary's full instance data with ReadInstance. If the read fails (e.g. the instance row is not yet in vtorc's backend or a DB error occurs), the error is wrapped with the primary's tablet alias. This guard prevents running errant-GTID diffing against unknown primary data.","triggerScenarios":"detectErrantGTIDs calling ReadInstance(primaryAlias) when the primary has not been discovered/read yet (newly promoted primary), or the underlying backend read returns an error.","commonSituations":"Immediately after a failover/promotion before vtorc's discovery has read the new primary, backend MySQL (sqlite/MySQL backing vtorc's data) issues, or a primary that was decommissioned while a replica still points at it.","solutions":["Retry on the next vtorc cycle — this is often transient right after promotion until the primary is discovered","Check vtorc logs for errors reading the primary tablet; verify the primary tablet is up and registered in the topo","Inspect the wrapped cause (%w) for backend DB errors vs 'instance not found'","Force a topology refresh of the shard or restart vtorc discovery if the primary stays unread"],"exampleFix":"// diagnosis aid\n// before\nreturn fmt.Errorf(\"failed to read primary instance %v: %w\", topoproto.TabletAliasString(primaryAlias), err)\n// after (operational fix, not code): ensure new primary tablets are readable\nvtctlclient GetTablet zone1-0000000100  # confirm tablet exists in topo before expecting vtorc reads to succeed","handlingStrategy":"retry","validationCode":"// Confirm the shard primary is known to vtorc before relying on errant-GTID detection\n// vtctlclient GetTablet <primary-alias>   # should succeed and show PRIMARY type","typeGuard":null,"tryCatchPattern":"err := runErrantGTIDCheck(ctx, tablet)\nif err != nil && strings.Contains(err.Error(), \"failed to read primary instance\") {\n    // common right after promotion; wait for next discovery cycle and retry\n    time.Sleep(discoveryInterval)\n    err = runErrantGTIDCheck(ctx, tablet)\n}","preventionTips":["Wait for post-promotion discovery to settle before trusting errant-GTID results","Keep primary tablets registered and healthy in the topo","Watch vtorc logs for repeated ReadInstance failures on the primary alias","Check vtorc's backend DB health (it stores instance data there)"],"tags":["vtorc","errant-gtid","topology","primary"],"backgroundTag":"topology-read-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}