vitessio/vitess · warning

aborting %s, primary mitigation is required

Error message

aborting %s, primary mitigation is required

What it means

Before executing a primary recovery, VTOrc re-checks the health of the analyzed primary. If the refreshed analysis no longer matches (the situation changed) or another VTOrc instance already mitigated it, the original analysis is considered stale and the recovery is deliberately aborted with this error so a fresh decision is made on the next cycle.

Source

Thrown at go/vt/vtorc/logic/topology_recovery.go:1245

	// checking if the original analysis is valid even after the primary refresh.
	alreadyFixed, _, err := checkIfAlreadyFixed(analysisEntry)
	if err != nil {
		log.Info(fmt.Sprintf("recheckPrimaryHealth: Checking if recovery is required returned err: %v", err))
		return err
	}

	if !alreadyFixed {
		return nil
	}

	// The original analysis for the tablet has changed.
	// This could mean that either the original analysis has changed or some other
	// VTOrc instance has already performing the mitigation.
	// In either case, the original analysis is stale which can be safely aborted.
	log.Info(fmt.Sprintf("recheckPrimaryHealth: Primary recovery is required, Tablet alias: %v", primaryTabletAlias))
	recoveriesSkippedCounter.Add(append(recoveryLabels, RecoverySkipPrimaryRecovery.String()), 1)
	return fmt.Errorf("aborting %s, primary mitigation is required", originalAnalysisEntry)
}

// checkIfAlreadyFixed checks whether the problem that the analysis entry represents has already been fixed by another agent or not.
// It returns (alreadyFixed, matchedEntry, error). When the problem still exists matchedEntry is the refreshed analysis entry
// that triggered the same recovery; callers can use it to re-evaluate policies (e.g. the cells-no-recovery cell gate)
// against the post-refresh state.
//
// Note: GetDetectionAnalysis may suppress non-primary analyses when a shard-wide
// action is detected. Problems that declare a dependency on the shard-wide action
// (via BeforeAnalyses/AfterAnalyses) survive suppression and will still be found
// here. Non-dependent problems are intentionally suppressed — the shard-wide
// action takes priority and they will be re-detected on a future poll.
func checkIfAlreadyFixed(analysisEntry *inst.DetectionAnalysis) (bool, *inst.DetectionAnalysis, error) {
	// Run a replication analysis again. We will check if the problem persisted
	analysisEntries, err := inst.GetDetectionAnalysis(analysisEntry.AnalyzedKeyspace, analysisEntry.AnalyzedShard, &inst.DetectionAnalysisHints{})
	if err != nil {
		return false, nil, err
	}

View on GitHub (pinned to 01a25a7d17)

Solutions

1. No action needed — this is a deliberate safety abort; VTOrc will re-analyze and start a new recovery if still required
2. Check the VTOrc/prometheus counter recoveries_skipped (RecoverySkipPrimaryRecovery) to confirm it was a benign skip
3. If it fires constantly, ensure only one VTOrc instance watches each keyspace or reduce recovery concurrency between instances
4. Investigate why the primary keeps needing mitigation if the abort repeats every cycle

if err := recover(...); err != nil && strings.Contains(err.Error(), "primary mitigation is required") {
    // benign abort: wait for next analysis cycle
    return nil
}

When it happens

Trigger: executeCheckAndRecoverFunction re-evaluates an analysis entry via recheckPrimaryHealth and finds the primary now requires its own mitigation (or the analysis changed), so the pending recovery for the original entry is abandoned.

Common situations: Running multiple VTOrc instances against the same cluster where one finished the repair first; the recovered tablet recovered on its own between detection and execution; stale analysis from before a failover completed.

Related errors

• failed to read tablet %q from vtorc db: %w
• could not find primary tablet %v
• failed to read instance: %w
• syslog is not supported on windows
• Log file number is zero, cannot detect previous file

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/cfadc4fa850f5beb. Report an issue: GitHub.