vitessio/vitess · error

failed to read instance: %w

Error message

failed to read instance: %w

What it means

At the start of executing a recovery function, VTOrc re-reads the analyzed tablet from its local DB (inst.ReadTablet) to operate on fresh data. If that read fails, the recovery is aborted and this wrapped error is returned so the failure is attributed to the recovery run instead of being silently ignored.

Source

Thrown at go/vt/vtorc/logic/topology_recovery.go:1576

		slog.String("tablet", aliasString),
	)

	// This has to be done in the end; whether successful or not, we should mark that the recovery is done.
	// So that after the active period passes, we are able to run other recoveries.
	defer func() {
		if err := resolveRecovery(topologyRecovery, nil); err != nil {
			logger.Error(
				"failed to resolve recovery",
				slog.String("recovery", ReconcileStaleTopoPrimaryRecoveryName),
				slog.Any("error", err),
			)
		}
	}()

	analyzedTablet, err := inst.ReadTablet(alias)
	if err != nil {
		logger.Error("failed to read tablet, aborting recovery", slog.String("tablet", aliasString))
		return false, topologyRecovery, fmt.Errorf("failed to read instance: %w", err)
	}

	var wg sync.WaitGroup

	// Make sure the best-effort steps complete or timeout before we return.
	defer wg.Wait()

	// On a best-effort basis, attempt to demote the tablet and configure replication concurrently
	// with the topology type update below. Failures here will not fail the overall recovery.
	wg.Go(func() {
		// Demote the tablet, forcing it to become read-only and drop pending transactions.
		if _, err := forceDemotePrimary(ctx, analyzedTablet); err != nil {
			logger.Error("failed to demote stale primary", slog.String("tablet", aliasString), slog.Any("error", err))
			return
		}

		logger.Info("successfully demoted stale primary", slog.String("tablet", aliasString))

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Check VTOrc backend DB health and restore connectivity; the recovery loop will retry automatically
2. Look at the wrapped error (%w) in VTOrc logs for the concrete SQL/DB failure
3. Verify schema of the vtorc instance tables after upgrades (vtorc has upgrade migrations)
4. Re-trigger the recovery manually via `vtorc ForcedCommand` or wait for the analysis cycle

if err := db.PingContext(ctx); err != nil { return err }

fixed, rec, err := executeCheckAndRecoverFunction(...)
if err != nil && strings.Contains(err.Error(), "failed to read instance") {
    // backend transient: let the next cycle retry
    return nil
}

When it happens

Trigger: Any recovery execution path reaches the analyzedTablet lookup with an alias and inst.ReadTablet returns a DB error (backend down, schema mismatch, connection limit).

Common situations: VTOrc backend MySQL unavailable mid-recovery; connection pool exhausted during failover storm; upgraded Vitess with old DB schema.

Related errors

• failed to read tablet %q from vtorc db: %w
• recovery.IsRecoveryDisabled(): %v
• unmarshal primary health state: %w
• marshal primary health state: %w
• could not find primary tablet %v

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/8191828e80888f72. Report an issue: GitHub.