vitessio/vitess · error

%v: %v

Error message

%v: %v

What it means

withRetry loops an RPC until success or context expiry; when ctx.Done() fires it returns a combined error of the context's error (e.g. context deadline exceeded / canceled) plus the last RPC error seen, so both the timeout and the underlying failure are visible.

Source

Thrown at go/vt/mysqlctl/grpcmysqlctlclient/client.go:154

		version = r.Version
		return nil
	})
	return version, err
}

// Close is part of the MysqlctlClient interface.
func (c *client) Close() {
	c.cc.Close()
}

// withRetry is needed because grpc doesn't handle some transient errors
// correctly (like EAGAIN) when sockets are used.
func (c *client) withRetry(ctx context.Context, f func() error) error {
	var lastError error
	for {
		select {
		case <-ctx.Done():
			return fmt.Errorf("%v: %v", ctx.Err(), lastError)
		default:
		}
		if err := f(); err != nil {
			if st, ok := status.FromError(err); ok {
				code := st.Code()
				if code == codes.Unavailable {
					lastError = err
					time.Sleep(100 * time.Millisecond)
					continue
				}
			}
			return err
		}
		return nil
	}
}

func init() {

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Increase the context deadline for long-running mysqlctl operations.
2. Verify the mysqlctl server is reachable and healthy (reduces Unavailable retries).
3. Inspect lastError in the combined message for the root RPC failure (e.g. connection refused).

Example fix

// before
ctx := context.Background(); defer cancel() // no timeout
// after
ctx, cancel := context.WithTimeout(t.Context(), 10*time.Minute)
defer cancel()

// pre-check reachability before long RPC
conn, err := net.DialTimeout("tcp", addr, 2*time.Second)
if err != nil { return fmt.Errorf("mysqlctl unreachable at %s", addr) }
conn.Close()

err := client.Start(ctx, ...)
if err != nil {
	if strings.Contains(err.Error(), context.DeadlineExceeded.Error()) {
		// increase deadline / check server health
	}
	return err
}

When it happens

Trigger: Calling any mysqlctl client RPC (Start, Shutdown, RunMysqlUpgrade, ApplyBinlogFile, ReadBinlogFilesTimestamps, ReinitConfig) whose retries exhaust past the caller's context deadline or get canceled.

Common situations: mysqlctl server down or slow so every attempt returns Unavailable and retries burn the whole deadline; caller-set context timeout too short for long operations like MysqlUpgrade; task cancellation during shutdown.

Related errors

• gRPC connection wait time exceeded
• failed to discover %ss (cluster %s): %w
• context was done before filtered replication did catch up. L
• invalid choice for enum
• deadline exceeded waiting for mysqld socket file to appear:

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/3cd5d0bbef1e4a72. Report an issue: GitHub.