{"record":{"id":"dcb73744e80b4e9c","repo":"apache/iceberg","slug":"connection-to-zookeeper-timed-out","errorCode":null,"errorMessage":"Connection to Zookeeper timed out","messagePattern":"Connection to Zookeeper timed out","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/ZkLockFactory.java","lineNumber":126,"sourceCode":"  @Override\n  public void open() {\n    if (isOpen) {\n      LOG.debug(\"ZkLockFactory already opened for lockId: {}.\", lockId);\n      return;\n    }\n\n    this.client =\n        CuratorFrameworkFactory.builder()\n            .connectString(connectString)\n            .sessionTimeoutMs(sessionTimeoutMs)\n            .connectionTimeoutMs(connectionTimeoutMs)\n            .retryPolicy(createRetryPolicy())\n            .build();\n    client.start();\n\n    try {\n      if (!client.blockUntilConnected(connectionTimeoutMs, TimeUnit.MILLISECONDS)) {\n        throw new IllegalStateException(\"Connection to Zookeeper timed out\");\n      }\n\n      this.taskSharedCount = new SharedCount(client, getTaskSharePath(), 0);\n      this.recoverySharedCount = new SharedCount(client, getRecoverySharedPath(), 0);\n      taskSharedCount.start();\n      recoverySharedCount.start();\n      isOpen = true;\n      LOG.info(\"ZkLockFactory initialized for lockId: {}.\", lockId);\n    } catch (InterruptedException e) {\n      Thread.currentThread().interrupt();\n      throw new RuntimeException(\"Interrupted while connecting to Zookeeper\", e);\n    } catch (Exception e) {\n      closeQuietly();\n      throw new RuntimeException(\"Failed to initialize SharedCount\", e);\n    }\n  }\n\n  private String getTaskSharePath() {","sourceCodeStart":108,"sourceCodeEnd":144,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/api/ZkLockFactory.java#L108-L144","documentation":"ZkLockFactory.open() starts a Curator client and waits blockUntilConnected(connectionTimeoutMs); if the ZooKeeper session is not established within the timeout, open() throws IllegalStateException with this message. The factory cannot be used without a live ZooKeeper connection.","triggerScenarios":"Calling open() when ZooKeeper is unreachable, slow, or overloaded and connectionTimeoutMs elapses before the session is established; also wrong zooKeeperAddress/host/port or the ensemble rejecting SASL/auth.","commonSituations":"Wrong quorum address or port (2181) in config; ZooKeeper ensemble down or in maintenance; firewall/Security Group blocking the port; connectionTimeoutMs set too low for a cross-datacenter ZooKeeper; DNS resolving to an unreachable host.","solutions":["Verify the zooKeeperAddress and port are correct and reachable (nc/zkCli from the Flink host).","Increase the connectionTimeoutMs value in ZkLockFactory.builder() if the network is slow.","Check ZooKeeper server health/logs (ensemble quorum may be lost).","Fix firewall/Security Group/DNS so the Flink node can reach the quorum.","Retry open() once the ensemble is healthy; keep retryPolicy generous for session recovery."],"exampleFix":"// before\nZkLockFactory.builder().setZooKeeperAddress(\"zk-internal:2181\") // unreachable\n  .setConnectionTimeoutMs(1000).build();\n// after\nZkLockFactory.builder().setZooKeeperAddress(\"zk-1:2181,zk-2:2181,zk-3:2181\")\n  .setConnectionTimeoutMs(15000).build();","handlingStrategy":"validation","validationCode":"// from the Flink host, before opening the factory\necho ruok | nc zk-1 2181  # expect imok","typeGuard":null,"tryCatchPattern":"try {\n  factory.open();\n} catch (IllegalStateException e) {\n  LOG.error(\"ZK connect timeout: {}\", e.getMessage());\n  // retry open with backoff or fail the trigger deployment\n}","preventionTips":["Verify ZooKeeper address/port reachability from Flink nodes","Set connectionTimeoutMs generously (>= 15s) for slow networks","Monitor ZooKeeper quorum health","Use the full quorum string, not a single server"],"tags":["flink","zookeeper","lock","timeout","curator"],"backgroundTag":"connection-timeout","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}