{"record":{"id":"f7bb4c6ee80a0e7e","repo":"apache/hadoop","slug":"cannot-get-children-for-znode-message","errorCode":null,"errorMessage":"Cannot get children for \"{znode}\": {message}","messagePattern":"Cannot get children for \"(.+?)\": (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreZooKeeperImpl.java","lineNumber":195,"sourceCode":"        for (Future<T> future : futures) {\n          if (future.get() != null) {\n            ret.add(future.get());\n          }\n        }\n      } else {\n        for (Callable<T> callable : callables) {\n          T record = callable.call();\n          if (record != null) {\n            ret.add(record);\n          }\n        }\n      }\n    } catch (Exception e) {\n      getMetrics().addFailure(monotonicNow() - start);\n      String msg = \"Cannot get children for \\\"\" + znode + \"\\\": \" +\n          e.getMessage();\n      LOG.error(msg);\n      throw new IOException(msg);\n    }\n    long end = monotonicNow();\n    getMetrics().addRead(end - start);\n    return new QueryResult<T>(ret, getTime());\n  }\n\n  /**\n   * Get one data record in the StateStore or delete it if it's corrupted.\n   *\n   * @param clazz Record class to evaluate.\n   * @param znode The ZNode for the class.\n   * @param child The child for znode to get.\n   * @return The record to get.\n   */\n  private <T extends BaseRecord> T getRecord(Class<T> clazz, String znode, String child) {\n    T record = null;\n    try {\n      String path = getNodePath(znode, child);","sourceCodeStart":177,"sourceCodeEnd":213,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/store/driver/impl/StateStoreZooKeeperImpl.java#L177-L213","documentation":"StateStoreZooKeeperImpl fetches records by listing a record znode's children and running per-child getRecord callables on an executor. Any exception during listing/execution is counted as a metrics failure and wrapped as IOException('Cannot get children for \"<znode>\": <cause message>'). Note the exception embeds only e.getMessage(), so the LOG.error line is the place to see the full stack; the znode name identifies which record class container failed.","triggerScenarios":"KeeperException.ConnectionLoss/SessionExpired/NoAuth while listing the record znode; NoNode when the record znode was deleted or never created; executor interrupted during shutdown; ACL mismatch denying reads on the state store znodes.","commonSituations":"ZooKeeper outage or session flaps behind the Router; state store znodes deleted out-of-band; ZK security enabled without matching auth on the router; router shutdown racing an in-flight fetch.","solutions":["Check the LOG.error line for the full underlying exception — the thrown IOException carries only the message.","Verify the znode from the message exists and is readable (zkCli.sh ls on that path with the router's auth).","Restore ZooKeeper connectivity/auth; the driver recovers on the next monitor cycle.","Restart the router if the failure occurred during initialization so the ZK driver re-establishes its session."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  QueryResult<T> r = zkDriver.fetchAll(clazz);\n} catch (IOException e) {\n  String m = e.getMessage();\n  if (m != null && m.startsWith(\"Cannot get children\")) {\n    // message tail carries the KeeperException text; check LOG.error for the stack\n    if (m.contains(\"ConnectionLoss\") || m.contains(\"SessionExpired\")) {\n      retryWithBackoff(); // transient ZK\n    } else {\n      throw e; // NoNode/NoAuth: provision or fix ACLs\n    }\n  } else { throw e; }\n}","preventionTips":["Verify state store znodes exist and are readable (zkCli ls) after first-time setup or ZK migrations.","Keep ZK ACL/auth configuration identical across routers; NoAuth shows up exactly here.","Alert on ZK session/connection metrics of the router process to catch flaps before record fetches fail."],"tags":["hdfs","router-based-federation","state-store","zookeeper"],"backgroundTag":"zookeeper-connection-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}