vitessio/vitess · critical

cannot dial to server %v: %v

Error message

cannot dial to server %v: %v

What it means

RunCommandAndWait dials the vtctl server using vtctlclient.New and wraps any dial/creation failure with 'cannot dial to server <addr>: <cause>'. It signals the client could not establish a connection (or was created at all) before running the command. The underlying cause (network refusal, unknown protocol, TLS) is preserved in %v.

Source

Thrown at go/vt/vtctl/vtctlclient/wrapper.go:41

	"io"
	"time"

	logutilpb "vitess.io/vitess/go/vt/proto/logutil"
)

var defaultTimeout = time.Hour

// RunCommandAndWait executes a single command on a given vtctld and blocks until the command did return or timed out.
// Output from vtctld is streamed as logutilpb.Event messages which
// have to be consumed by the caller who has to specify a "recv" function.
func RunCommandAndWait(ctx context.Context, server string, args []string, recv func(*logutilpb.Event)) error {
	if recv == nil {
		return errors.New("no function closure for Event stream specified")
	}
	// create the client
	client, err := New(ctx, server)
	if err != nil {
		return fmt.Errorf("cannot dial to server %v: %v", server, err)
	}
	defer client.Close()

	// run the command ( get the timeout from the context )
	timeout := defaultTimeout
	deadline, ok := ctx.Deadline()
	if ok {
		timeout = time.Until(deadline)
	}
	stream, err := client.ExecuteVtctlCommand(ctx, args, timeout)
	if err != nil {
		return fmt.Errorf("cannot execute remote command: %v", err)
	}

	// stream the result
	for {
		e, err := stream.Recv()
		switch err {

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Verify the vtctl server address and port are correct and the server process is healthy (curl/grpc probe the endpoint)
2. Check network reachability: firewall rules, k8s Service/Endpoints, DNS resolution for the addr
3. Confirm the vtctl client protocol flag matches the server and the implementation is linked (see 'unknown vtctl client protocol')
4. Restart or redeploy vtctld if it crashed, then retry the command

Example fix

// before
vtctlclient.RunCommandAndWait(ctx, "localhost:15999", args, nil, true)
// after
// confirm server is up, then use the real address
vtctlclient.RunCommandAndWait(ctx, "vtctld.example.com:15999", args, nil, true)

conn, err := net.DialTimeout("tcp", addr, 2*time.Second)
if err != nil {
  return fmt.Errorf("vtctl server unreachable at %s: %w", addr, err)
}
conn.Close()

err := vtctlclient.RunCommandAndWait(ctx, server, args, recv, true)
if err != nil && strings.Contains(err.Error(), "cannot dial to server") {
  // backoff and retry once server health passes
  time.Sleep(time.Second)
  err = vtctlclient.RunCommandAndWait(ctx, server, args, recv, true)
}

When it happens

Trigger: Calling vtctlclient.RunCommandAndWait(ctx, server, args, recv, ...) when the vtctld/vtctlserver address is wrong, the server is down, the port is closed, or New() fails due to an unregistered protocol.

Common situations: vtctld not running or crashed; wrong -vtctl_server address/port; firewall or k8s service misrouting; DNS failure; server upgraded while client uses incompatible protocol.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• cannot execute remote command: %v
• error dialing binlog server: %v
• remote error: %v
• failed to load static auth plugin. Plugin configured but grp
• gRPC connection wait time exceeded

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/4d6b4098ff29932f. Report an issue: GitHub.