t8y2/dbx · error

Connection timed out

Error message

Connection timed out

What it means

openClient watches the session event channel after zk.Connect and waits for zk.StateHasSession. If the event channel closes before a session is established (the zk driver shut down its event loop), or if the overall connectionTimeout timer fires first, the connection is closed and this 'Connection timed out' error is returned.

Source

Thrown at agents/drivers/zookeeper/connection.go:213

	connection, events, err := zk.Connect(
		target.Servers,
		sessionTimeout,
		zk.WithDialer(dialer),
		zk.WithLogInfo(false),
		zk.WithMaxBufferSize(maxBufferSize),
	)
	if err != nil {
		return nil, err
	}
	connected := false
	timer := time.NewTimer(connectionTimeout)
	defer timer.Stop()
	for !connected {
		select {
		case event, open := <-events:
			if !open {
				connection.Close()
				return nil, errors.New("Connection timed out")
			}
			if event.State == zk.StateHasSession {
				connected = true
			}
			if event.State == zk.StateAuthFailed {
				connection.Close()
				return nil, errors.New("ZooKeeper authentication failed")
			}
		case <-timer.C:
			connection.Close()
			return nil, errors.New("Connection timed out")
		}
	}

	if authScheme == defaultAuthScheme && strings.TrimSpace(config.Username) != "" {
		credentials := []byte(strings.TrimSpace(config.Username) + ":" + config.Password)
		if err := connection.AddAuth(defaultAuthScheme, credentials); err != nil {
			connection.Close()

View on GitHub (pinned to c0390bff16)

Solutions

1. Verify the ZooKeeper ensemble is reachable and healthy (zkServer.sh status / ruok four-letter word on each host).
2. Increase the connectionTimeout config value (ConnectionTimeoutMS) so slow-but-working ensembles have time to grant a session.
3. Check network paths/firewalls between the client and the listed servers; silent packet drops stall the session handshake.
4. Confirm the connect string lists valid host:port entries and try connecting manually (e.g. nc host 2181).

Example fix

// before
connCfg.ConnectionTimeoutMS = 2000
// after
connCfg.ConnectionTimeoutMS = 15000

// Pre-flight reachability check before connecting
conn, err := net.DialTimeout("tcp", host+":2181", 5*time.Second)
if err != nil {
    return fmt.Errorf("zookeeper host %s unreachable: %w", host, err)
}
conn.Close()

var session *ClientSession
err := retry.Do(3, 2*time.Second, func() error {
    s, err := openClient(cfg)
    if err != nil {
        if strings.Contains(err.Error(), "Connection timed out") {
            return retry.Retryable(err) // transient network issue
        }
        return err // non-retryable
    }
    session = s
    return nil
})

When it happens

Trigger: zk.Connect succeeds at the TCP level but no StateHasSession event arrives within config.ConnectionTimeoutMS; or the zk event channel closes prematurely while the for/select loop is still waiting. Raised from openClient in agents/drivers/zookeeper/connection.go:213.

Common situations: ZooKeeper ensemble unreachable or overloaded (server up but slow to grant sessions); network firewall dropping packets silently (SYN succeeds, session handshake stalls); connection timeout configured too low for a distant/lossy network; DNS resolving to an unresponsive host.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• ZooKeeper connection timed out before a session was establis
• ZooKeeper connection event: %w
• ZooKeeper connection timed out before a session was establis
• ZooKeeper session expired during connection
• read ZooKeeper connect response: %w

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/41704a3fea365bfc. Report an issue: GitHub.