apache/pulsar · error · RuntimeException

Error reading list

Error message

Error reading list

What it means

Unchecked RuntimeException thrown from the hasNext() of the iterator returned by listLedgersToRereplicate when fetching children or underreplication info from the metadata store fails (any non-interrupt Exception, including ExecutionException/TimeoutException from the blocking .get() call). Because it surfaces lazily while iterating, it escapes Iterator.hasNext()/next() rather than the listLedgersToRereplicate call itself.

Source

Thrown at pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java:542

                            String child = parent + "/" + c;
                            if (c.startsWith("urL")) {
                                long ledgerId = getLedgerId(child);
                                UnderreplicatedLedger underreplicatedLedger = getLedgerUnreplicationInfo(ledgerId);
                                if (underreplicatedLedger != null) {
                                    List<String> replicaList = underreplicatedLedger.getReplicaList();
                                    if ((predicate == null) || predicate.test(replicaList)) {
                                        curBatch.add(underreplicatedLedger);
                                    }
                                }
                            } else {
                                queue.add(child);
                            }
                        }
                    } catch (InterruptedException ie) {
                        Thread.currentThread().interrupt();
                        return false;
                    } catch (Exception e) {
                        throw new RuntimeException("Error reading list", e);
                    }
                }
                return curBatch.size() > 0;
            }

            @Override
            public UnderreplicatedLedger next() {
                assert curBatch.size() > 0;
                return curBatch.remove();
            }
        };
    }

    private long getLedgerToRereplicateFromHierarchy(String parent, long depth)
            throws ExecutionException, InterruptedException, TimeoutException {
        if (depth == 4) {
            List<String> children = new ArrayList<>(store.getChildren(parent)
                    .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS));

View on GitHub (pinned to 820761864e)

Solutions

1. Inspect the RuntimeException cause for the underlying MetadataStoreException to identify timeout vs connection loss vs not-found.
2. Verify the underreplication hierarchy under /underreplication is intact (no manual znode deletions).
3. Catch the RuntimeException where you consume the iterator and rescan (the iterator is not resumable after failure).
4. Ensure metadata store health/latency is within BLOCKING_CALL_TIMEOUT; scale or tune the ensemble if scans repeatedly time out.

Example fix

// before: letting the RuntimeException propagate and kill the auditor scan
Iterator<UnderreplicatedLedger> it = mgr.listLedgersToRereplicate(null);
while (it.hasNext()) { process(it.next()); }

// after: guard the iteration
try {
    Iterator<UnderreplicatedLedger> it = mgr.listLedgersToRereplicate(null);
    while (it.hasNext()) { process(it.next()); }
} catch (RuntimeException e) {
    log.warn("Scan of underreplicated ledgers failed, will retry", e);
}

// Java: preflight the hierarchy root before iterating
try {
    store.getChildren(urLedgerPath).get(5, TimeUnit.SECONDS);
} catch (Exception e) {
    throw new IllegalStateException("Underreplication hierarchy unreadable", e);
}

try {
    Iterator<UnderreplicatedLedger> it = manager.listLedgersToRereplicate(predicate);
    while (it.hasNext()) {
        process(it.next());
    }
} catch (RuntimeException e) {
    // 'Error reading list': inspect e.getCause() (MetadataStoreException) and rescan
    log.warn("Underreplicated ledger listing failed", e);
}

When it happens

Trigger: Iterating the iterator from listLedgersToRereplicate(predicate) when: store.getChildren(parent).get() times out (BLOCKING_CALL_TIMEOUT exceeded) or fails; or getLedgerUnreplicationInfo(ledgerId) throws for a child urL znode; or the parent hierarchy path is missing/corrupt in the metadata store.

Common situations: ZooKeeper connection loss or session expiry mid-iteration over a large underreplicated-ledger tree; hierarchy cleanup races deleting a parent node between listing and read; metadata store latency exceeding the blocking timeout during auditor scans after many bookie failures.

Related errors

• Failed to get children of ${path}
• Failed to check exist ${POLICIES_READONLY_FLAG_PATH}
• Error preloading next range
• Error when get child nodes from zk
• Error contacting with metadata store

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/70ccc01ef0d085c4. Report an issue: GitHub.