apache/pulsar · error · IOException

IOException

Error message

IOException

What it means

validateAndNukeExistingCluster lists the children of the ledgers root path by doing a blocking get() with a timeout on the metadata store future. If the store call fails asynchronously (ExecutionException) or does not complete within BLOCKING_CALL_TIMEOUT (TimeoutException), the failure is wrapped in an IOException and thrown.

Source

Thrown at pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerManagerFactory.java:124

    }

    @Override
    public boolean validateAndNukeExistingCluster(AbstractConfiguration<?> conf,
                                                  LayoutManager layoutManager)
            throws InterruptedException, IOException {
        @Cleanup
        PulsarLedgerManager ledgerManager = new PulsarLedgerManager(store, ledgerRootPath);

        /*
         * before proceeding with nuking existing cluster, make sure there
         * are no unexpected nodes under ledgersRootPath
         */
        final List<String> ledgersRootPathChildrenList;
        try {
            ledgersRootPathChildrenList = store.getChildren(ledgerRootPath)
                    .get(BLOCKING_CALL_TIMEOUT, TimeUnit.MILLISECONDS);
        } catch (ExecutionException | TimeoutException e) {
            throw new IOException(e);
        }
        for (String ledgersRootPathChildren : ledgersRootPathChildrenList) {
            if ((!AbstractZkLedgerManager.isSpecialZnode(ledgersRootPathChildren))
                    && (!ledgerManager.isLedgerParentNode(ledgersRootPathChildren))) {
                log.error().attr("node", ledgersRootPathChildren).attr("ledgersRootPath", ledgerRootPath)
                        .log("Found unexpected node under ledgersRootPath, exiting nuke operation");
                return false;
            }
        }

        // formatting ledgermanager deletes ledger znodes
        format(conf, layoutManager);

        // now delete all the special nodes recursively
        final List<String> ledgersRootPathChildren;
        try {
            ledgersRootPathChildren = store.getChildren(ledgerRootPath)
                    .get(BLOCKING_CALL_TIMEOUT, TimeUnit.MILLISECONDS);

View on GitHub (pinned to 820761864e)

Solutions

  1. Verify metadata store connectivity and health before re-running the nuke operation.
  2. Inspect the wrapped cause (e.getCause()) for the real store error and address it (auth, session expiry, no quorum).
  3. Retry the operation once the store is responsive; nuke is idempotent up to the failed step.
  4. If listings are slow, reduce store load or increase capacity; check for large znode trees.
Defensive patterns

Strategy: retry

Validate before calling

// pre-check store reachability
store.exists(ledgerRootPath).get(BLOCKING_CALL_TIMEOUT, TimeUnit.MILLISECONDS);

Try / catch

try {
    validateAndNukeExistingCluster(...);
} catch (IOException e) {
    Throwable cause = e.getCause();
    if (cause instanceof TimeoutException) { /* retry after store recovers */ }
}

Prevention

When it happens

Trigger: Running the nuke-cluster operation when the metadata store is unreachable, returns an error for getChildren on the ledgers root, or is too slow (network latency, overloaded store, GC pauses) to answer within the blocking timeout.

Common situations: ZooKeeper/MetadataStore outage or quorum loss during a cluster teardown; network partition; very large ledgers root making listing slow; wrong connection settings causing hangs.

Related errors


AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/0d029cdaf6957337. Report an issue: GitHub.