vitessio/vitess · error

failed to transition from state %s

Error message

failed to transition from state %s

What it means

grpcvtctldclient.WaitForReady polls the gRPC channel state while trying to establish a connection to the vtctld server. If WaitForStateChange returns false — meaning the context was cancelled or the channel shut down — before reaching READY, it reports 'failed to transition from state %s'. This surfaces connection establishment failures (unreachable server, dial errors) as a state-transition error.

Source

Thrown at go/vt/vtctl/grpcvtctldclient/client.go:124

			connState := client.cc.GetState()

			switch connState {
			case connectivity.Ready:
				return nil

			// Per https://github.com/grpc/grpc/blob/master/doc/connectivity-semantics-and-api.md,
			// a client that enters the SHUTDOWN state never leaves this state, and all new RPCs should
			// fail immediately. Further polling is futile, in other words, and so we
			// return an error immediately to indicate that the caller can close the connection.
			case connectivity.Shutdown:
				return ErrConnectionShutdown

			// If the connection is IDLE, CONNECTING, or in a TRANSIENT_FAILURE mode,
			// then we wait to see if it will transition to a READY state.
			default:
				if !client.cc.WaitForStateChange(ctx, connState) {
					// If the client has failed to transition, fail so that the caller can close the connection.
					return fmt.Errorf("failed to transition from state %s", connState)
				}
			}
		}
	}
}

func init() {
	vtctldclient.Register("grpc", gRPCVtctldClientFactory)
}

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Verify the vtctld server is running and the address/port is correct (e.g. `vtctldclient --server host:port GetSrvKeyspaces`).
2. Check the ctx deadline/cancellation and increase the timeout if needed.
3. Inspect gRPC TRANSIENT_FAILURE cause (TLS certs, DNS) in server/client logs.
4. Recreate the client if the ClientConn was closed by mistake.

Example fix

// before
ctx := context.Background()
client.WaitForReady(ctx) // hangs or fails on dead server
// after
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := client.WaitForReady(ctx); err != nil {
    log.Warn("vtctld not ready", slog.Any("error", err))
}

// pre-check connectivity before WaitForReady
conn, err := grpc.NewClient(addr, opts...)
if err != nil { return err }
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
_ = conn

func isTransitionFailure(err error) bool {
    return err != nil && strings.Contains(err.Error(), "failed to transition from state")
}

if err := client.WaitForReady(ctx); err != nil {
    if isTransitionFailure(err) {
        // check vtctld is up, then retry with backoff
        return retryWithBackoff(ctx, client.WaitForReady)
    }
    return err
}

When it happens

Trigger: Dialing a vtctld address that is down or wrong; context timeout/cancellation while the channel is IDLE, CONNECTING, or TRANSIENT_FAILURE; the ClientConn being closed during WaitForReady.

Common situations: vtctld not running or wrong port in VTDATARUN/vtctld config; network/firewall blocking the gRPC port; tests cancelling contexts too early; TLS mismatch causing TRANSIENT_FAILURE.

Related errors

• GetKeyspaces(cluster = %s): %w
• error creating vtctldclient: %w
• uncaught panic: %v from: %v
• invalid key:value pair
• parse error

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/5c497d242b4564ce. Report an issue: GitHub.