{"record":{"id":"04d5f537a70c34c9","repo":"apache/pulsar","slug":"error-when-get-child-nodes-from-zk","errorCode":null,"errorMessage":"Error when get child nodes from zk","messagePattern":"Error when get child nodes from zk","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/LegacyHierarchicalLedgerRangeIterator.java","lineNumber":171,"sourceCode":"     * @param level1\n     *          1st level node name\n     * @param level2\n     *          2nd level node name\n     * @throws IOException\n     */\n    LedgerManager.LedgerRange getLedgerRangeByLevel(final String level1, final String level2)\n            throws IOException {\n        StringBuilder nodeBuilder = threadLocalNodeBuilder.get();\n        nodeBuilder.setLength(0);\n        nodeBuilder.append(ledgersRoot).append(\"/\")\n                .append(level1).append(\"/\").append(level2);\n        String nodePath = nodeBuilder.toString();\n        List<String> ledgerNodes = null;\n        try {\n            ledgerNodes = store.sync(nodePath).thenCompose(__ -> store.getChildrenFromStore(nodePath))\n                    .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS);\n        } catch (ExecutionException | TimeoutException e) {\n            throw new IOException(\"Error when get child nodes from zk\", e);\n        } catch (InterruptedException e) {\n            Thread.currentThread().interrupt();\n            throw new IOException(\"Error when get child nodes from zk\", e);\n        }\n        NavigableSet<Long> zkActiveLedgers =\n                HierarchicalLedgerUtils.ledgerListToSet(ledgerNodes, ledgersRoot, nodePath);\n        log.debug().attr(\"hashNode\", level1 + \"/\" + level2)\n                .attr(\"ledgers\", zkActiveLedgers)\n                .log(\"All active ledgers from ZK for hash node\");\n\n        return new LedgerManager.LedgerRange(zkActiveLedgers.subSet(getStartLedgerIdByLevel(level1, level2), true,\n                getEndLedgerIdByLevel(level1, level2), true));\n    }\n\n    /**\n     * Get the smallest cache id in a specified node /level1/level2.\n     *\n     * @param level1","sourceCodeStart":153,"sourceCodeEnd":189,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/LegacyHierarchicalLedgerRangeIterator.java#L153-L189","documentation":"IOException thrown by LegacyHierarchicalLedgerRangeIterator.getLedgerRangeByLevel when the sync+getChildrenFromStore call to the metadata store fails with ExecutionException or TimeoutException while fetching the child nodes of a hash node in the hierarchical ledger layout. It means the ZK-style metadata store could not be read (failure or too slow).","triggerScenarios":"preload() -> getLedgerRangeByLevel for an L1/L2 node; the underlying store future completes exceptionally or exceeds BLOCKING_CALL_TIMEOUT.","commonSituations":"ZooKeeper session expiry or quorum loss; network timeouts; very large child-node lists exceeding the store's response limits or the blocking timeout.","solutions":["Check ZooKeeper/metadata-store health and network connectivity","Increase BLOCKING_CALL_TIMEOUT for large hierarchies","Inspect the wrapped cause (e.getCause()) to distinguish store errors from timeouts","Retry enumeration once the store is healthy"],"exampleFix":"// before\nledgerNodes = store.sync(nodePath).thenCompose(__ -> store.getChildrenFromStore(nodePath))\n        .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS); // IOException on failure\n// after\ntry {\n    ledgerNodes = store.sync(nodePath).thenCompose(__ -> store.getChildrenFromStore(nodePath))\n            .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS);\n} catch (ExecutionException e) {\n    LOG.error(\"Failed to read children of {} from metadata store\", nodePath, e.getCause());\n    throw new IOException(\"Error when get child nodes from zk\", e);\n}","handlingStrategy":"retry","validationCode":"store.sync(ledgersRoot).orTimeout(5, TimeUnit.SECONDS).join(); // verify store reachable first","typeGuard":null,"tryCatchPattern":"try {\n    it.forEachRemaining(this::process);\n} catch (IOException e) {\n    if (e.getMessage().equals(\"Error when get child nodes from zk\")) {\n        LOG.error(\"Metadata store read failed for ledger hierarchy\", e.getCause()); // retry/backoff\n    }\n    throw e;\n}","preventionTips":["Monitor ZK session health and quorum before heavy enumeration","Tune BLOCKING_CALL_TIMEOUT for cluster size and network latency","Retry with backoff; distinguish ExecutionException cause from timeout"],"tags":["metadata-store","zookeeper","timeout"],"backgroundTag":"metadata-store-timeout","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}