t8y2/dbx · critical

No reachable ZooKeeper server within %dms: %s

Error message

No reachable ZooKeeper server within %dms: %s

What it means

requireReachableServer probes the configured server endpoints under a timeout context; if none of them accept a TCP connection before the deadline or the context is cancelled, the driver aborts connection setup with this error listing the timeout and all attempted servers.

Source

Thrown at agents/drivers/zookeeper/connection.go:429

			select {
			case jobs <- server:
			case <-ctx.Done():
				return
			}
		}
	}()
	done := make(chan struct{})
	go func() {
		waitGroup.Wait()
		close(done)
	}()
	select {
	case <-reachable:
		return nil
	case <-done:
	case <-ctx.Done():
	}
	return fmt.Errorf("No reachable ZooKeeper server within %dms: %s", timeout.Milliseconds(), strings.Join(servers, ","))
}

func endpointAddress(endpoint string) (string, error) {
	value := strings.TrimSpace(endpoint)
	if strings.HasPrefix(value, "[") {
		if _, _, err := net.SplitHostPort(value); err == nil {
			return value, nil
		}
		return value + ":" + strconv.Itoa(defaultPort), nil
	}
	if strings.Count(value, ":") == 0 {
		return net.JoinHostPort(value, strconv.Itoa(defaultPort)), nil
	}
	if strings.Count(value, ":") == 1 {
		if _, _, err := net.SplitHostPort(value); err != nil {
			return "", err
		}
		return value, nil

View on GitHub (pinned to c0390bff16)

Solutions

  1. Verify each server address and port (default 2181) is correct and reachable: nc -zv host 2181.
  2. Check the ZooKeeper ensemble is running (zServer status) and not in a failed/quorum-lost state.
  3. Increase the connection timeout if the network is slow or servers are slow to accept.
  4. Check firewalls/VPNs/security groups between client and servers; confirm DNS resolves the hostnames.

Example fix

// before
dsn := "zookeeper://zk1.internal:2182,zk2.internal:2182"
// after
dsn := "zookeeper://zk1.internal:2181,zk2.internal:2181" // correct client port
Defensive patterns

Strategy: retry

Validate before calling

for _, ep := range strings.Split(serverList, ",") {
	conn, err := net.DialTimeout("tcp", normalizeEndpoint(ep), 3*time.Second)
	if err != nil {
		return fmt.Errorf("precheck: %s unreachable: %v", ep, err)
	}
	conn.Close()
}

Try / catch

var sess *clientSession
err := retry.Do(3, 2*time.Second, func() error {
	s, err := connect(cfg)
	if err != nil && strings.Contains(err.Error(), "No reachable ZooKeeper server") {
		return retry.Retryable(err) // transient network; retry with backoff
	}
	if err != nil {
		return err
	}
	sess = s
	return nil
})

Prevention

When it happens

Trigger: Calling connect/openClient where every host:port in the server list is unreachable (connection refused, DNS failure, firewall) or too slow, within the configured connection timeout.

Common situations: Wrong host or port in the connection string, ZooKeeper ensemble down or restarting, network/firewall blocking the client port (2181), or timeout set too low for a slow network.

Related errors


AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/f0109c749cad442c. Report an issue: GitHub.