t8y2/dbx · error · IllegalStateException

Timed out connecting to ZooKeeper for Kafka broker discovery

Error message

Timed out connecting to ZooKeeper for Kafka broker discovery

What it means

Thrown by discoverBootstrapServers when a ZooKeeper client did not reach the SyncConnected state within zookeeper_connection_timeout_ms (default DEFAULT_ZOOKEEPER_CONNECTION_TIMEOUT_MS). The ensemble is unreachable, slow, or the connect string is wrong, so broker discovery via ZooKeeper cannot proceed.

Source

Thrown at agents/drivers/kafka/src/main/java/com/dbx/agent/kafka/KafkaAgent.java:349

        resolved.addProperty("bootstrap_servers", discoverBootstrapServers(connectString, securityProtocol(conn), conn));
        return resolved;
    }

    private static String discoverBootstrapServers(String connectString, String securityProtocol, JsonObject conn)
        throws Exception {
        int sessionTimeout = intOrDefault(conn, "zookeeper_session_timeout_ms", DEFAULT_SESSION_TIMEOUT_MS);
        int connectionTimeout = intOrDefault(
            conn,
            "zookeeper_connection_timeout_ms",
            DEFAULT_ZOOKEEPER_CONNECTION_TIMEOUT_MS
        );
        CountDownLatch connected = new CountDownLatch(1);
        ZooKeeper zooKeeper = new ZooKeeper(connectString, sessionTimeout, event -> {
            if (event.getState() == Watcher.Event.KeeperState.SyncConnected) connected.countDown();
        }, zooKeeperClientConfig(conn));
        try {
            if (!connected.await(connectionTimeout, TimeUnit.MILLISECONDS)) {
                throw new IllegalStateException("Timed out connecting to ZooKeeper for Kafka broker discovery");
            }

            List<String> brokerIds;
            try {
                brokerIds = new ArrayList<>(zooKeeper.getChildren("/brokers/ids", false));
            } catch (KeeperException.NoNodeException e) {
                throw new IllegalStateException("ZooKeeper path /brokers/ids does not exist", e);
            }
            brokerIds.sort(KafkaAgent::compareBrokerIds);

            List<JsonObject> registrations = new ArrayList<>();
            for (String brokerId : brokerIds) {
                try {
                    byte[] data = zooKeeper.getData("/brokers/ids/" + brokerId, false, null);
                    registrations.add(JsonParser.parseString(new String(data, StandardCharsets.UTF_8)).getAsJsonObject());
                } catch (KeeperException.NoNodeException e) {
                    // Expected race: a broker may refresh its ephemeral node between list and read.
                    logger().debug("Kafka broker {} disappeared during ZooKeeper discovery", brokerId);

View on GitHub (pinned to c0390bff16)

Solutions

1. Verify the zookeeper_connect_string host:port entries are reachable from the agent host
2. Increase zookeeper_connection_timeout_ms and zookeeper_session_timeout_ms
3. Check network/firewall rules and ZooKeeper ensemble health (four-letter words like srvr)
4. If ZooKeeper is not actually fronting this Kafka cluster, configure bootstrap_servers directly instead

When it happens

Trigger: Thrown at agents/drivers/kafka/src/main/java/com/dbx/agent/kafka/KafkaAgent.java:349 when the library encounters an invalid state.

Common situations: See trigger scenarios.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/60da8c1cd0286421. Report an issue: GitHub.