vitessio/vitess · error

tablet %v InitReplica failed: %v

Error message

tablet %v InitReplica failed: %v

What it means

When re-initializing replicas to point at the new primary, the InitReplica RPC to a tablet failed. Each failing replica is recorded and reported; the shard record may still have been updated to the new primary, so remaining replicas need fixing.

Source

Thrown at go/vt/vtctl/grpcvtctldserver/server.go:2980

	wgReplicas := sync.WaitGroup{}
	var primaryErr error
	for alias, tabletInfo := range tabletMap {
		if alias == primaryElectTabletAliasStr {
			wgPrimary.Add(1)
			go func(alias string, tabletInfo *topo.TabletInfo) {
				defer wgPrimary.Done()
				logger.Infof("populating reparent journal on new primary %v", alias)
				primaryErr = tmc.PopulateReparentJournal(replCtx, tabletInfo.Tablet, now,
					initShardPrimaryOperation,
					req.PrimaryElectTabletAlias, rp)
			}(alias, tabletInfo)
		} else {
			wgReplicas.Add(1)
			go func(alias string, tabletInfo *topo.TabletInfo) {
				defer wgReplicas.Done()
				logger.Infof("initializing replica %v", alias)
				if err := tmc.InitReplica(replCtx, tabletInfo.Tablet, req.PrimaryElectTabletAlias, rp, now, policy.IsReplicaSemiSync(durability, primaryElectTabletInfo.Tablet, tabletInfo.Tablet)); err != nil {
					rec.RecordError(fmt.Errorf("tablet %v InitReplica failed: %v", alias, err))
				}
			}(alias, tabletInfo)
		}
	}

	// After the primary is done, we can update the shard record
	// (note with semi-sync, it also means at least one replica is done).
	wgPrimary.Wait()
	if primaryErr != nil {
		// The primary failed, there is no way the
		// replicas will work.  So we cancel them all.
		logger.Warningf("primary failed to PopulateReparentJournal, canceling replicas")
		replCancel()
		wgReplicas.Wait()
		return fmt.Errorf("failed to PopulateReparentJournal on primary: %v", primaryErr)
	}
	if !topoproto.TabletAliasEqual(shardInfo.PrimaryAlias, req.PrimaryElectTabletAlias) {
		if _, err := s.ts.UpdateShardFields(ctx, req.Keyspace, req.Shard, func(si *topo.ShardInfo) error {

View on GitHub (pinned to 01a25a7d17)

Solutions

  1. Check the reported tablet's vttablet/mysqld logs and fix replication (START REPLICA, reset then retry)
  2. Run `vtctldclient InitReplica`-equivalent (ReparentTablet) on the failing tablet individually
  3. Confirm the tablet is serving from the right keyspace/shard and not deprecated/scraped
  4. If unrecoverable, Scrap and re-provision the tablet

Example fix

// before: replica can't connect to new primary
replica> SHOW REPLICA STATUS;  # wrong master host
// after: point replica at the new primary manually then retry
replica> CHANGE REPLICATION SOURCE TO SOURCE_HOST='<new-primary>', ...;
replica> START REPLICA;
Defensive patterns

Strategy: retry

Validate before calling

// pre-flight: every replica must be reachable and not scraped
for _, t := range shardTablets {
    if t.Type == topodatapb.TabletType_SCRAP { continue }
    if err := pingTablet(ctx, t); err != nil {
        return fmt.Errorf("replica %s unhealthy: %w", t.Alias, err)
    }
}

Try / catch

// collect failing replicas and remediate individually
if strings.Contains(err.Error(), "InitReplica failed") {
    for _, alias := range parseFailedAliases(err) {
        repointReplicaManually(ctx, alias, newPrimary)
    }
}

Prevention

When it happens

Trigger: InitShardPrimary loop calling tmc.InitReplica on each replica — tablet down, mysql replication cannot be started, or wrong primary alias passed.

Common situations: A replica is in a bad replication state (e.g. old primary still claiming); tablet agent unreachable; durability/semi-sync policy prevents starting replication against the new primary.

Related errors


AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/11ddee5455a7d686. Report an issue: GitHub.