vitessio/vitess · error

ReplicationStatus(%s) failed: %s

Error message

ReplicationStatus(%s) failed: %s

What it means

Aggregated error recorded when fetching a replica's ReplicationStatus fails for reasons other than context cancellation or deadline exceeded during shard replication status gathering. Like the PrimaryPosition case, a non-timeout error is fatal for the whole request.

Source

Thrown at go/vt/vtctl/grpcvtctldserver/server.go:4252

				span, ctx := trace.NewSpan(ctx, "VtctldServer.getReplicationStatus")
				defer span.Finish()

				span.Annotate("tablet_alias", alias)

				ctx, cancel := context.WithTimeout(ctx, topo.RemoteOperationTimeout)
				defer cancel()

				status, err := s.tmc.ReplicationStatus(ctx, tablet)
				if err != nil {
					switch ctx.Err() {
					case context.Canceled:
						log.Warn(fmt.Sprintf("context canceled before obtaining replication position from %s: %s", alias, err))
					case context.DeadlineExceeded:
						log.Warn(fmt.Sprintf("context deadline exceeded before obtaining replication position from %s: %s", alias, err))
					default:
						// The RPC was not timed out or canceled. We treat this
						// as a fatal error for the overall request.
						rec.RecordError(fmt.Errorf("ReplicationStatus(%s) failed: %s", alias, err))
						return
					}

					status = nil // Don't record any position for this tablet.
				}

				m.Lock()
				defer m.Unlock()

				results[alias] = status
				tabletMap[alias] = tablet
			}(ctx, alias, tabletInfo.Tablet)
		}
	}

	wg.Wait()

	if rec.HasErrors() {

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Check that all tablets in the shard are serving and MySQL is up
2. Verify connectivity to each tablet's tabletmanager gRPC port
3. Inspect the named tablet's logs for the underlying ReplicationStatus failure
4. Retry once the tablet recovers; replication status gathering is transient-sensitive

Example fix

// before
default:
    // The RPC was not timed out or canceled.
    rec.RecordError(fmt.Errorf("ReplicationStatus(%s) failed: %s", alias, err))
    return
// after
default:
    // The RPC was not timed out or canceled.
    rec.RecordError(fmt.Errorf("ReplicationStatus(%s) failed: %w", alias, err))
    return

// verify replica tablets are serving before gathering statuses
for alias in shardTablets { _, err := tmClient.Ping(ctx, alias); if err != nil { skip(alias) } }

try {
  statuses = await client.getShardReplicationStatus(keyspace, shard)
} catch (e) {
  if (String(e).includes('ReplicationStatus')) {
    // one tablet failed fatally; identify it from the error and check its health
    checkTabletHealth(parseAlias(e)); await backoffRetry(2)
  } else { throw e }
}

When it happens

Trigger: Shard replication status request where a tablet's ReplicationStatus tabletmanager RPC fails (connection refused, tablet down, gRPC error), and the error is not Canceled or DeadlineExceeded.

Common situations: A replica tablet is stopped mid-request; MySQL is down on the replica so SHOW REPLICA STATUS fails; network partition between vtctld and the tablet.

Related errors

• PrimaryPosition(%s) failed: %w
• no rpl_semi_sync_replica_status variable in mysql
• tablet %v InitReplica failed: %v
• ReplicationStatus(%v) failed: %v
• failed to ensure replication was started on tablet %s after

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/3fdc15a1071e41b8. Report an issue: GitHub.