apache/pulsar · error · MetadataStoreException

${name} failed to fill existing items in ${secs} secs. Fille

Error message

${name} failed to fill existing items in ${secs} secs. Filled count:${count}

What it means

MetadataStoreException thrown by MetadataStoreTableViewImpl.fill when iterating and loading all existing items under the tableview root does not complete within maxWaitTime milliseconds. The wrapped cause (InterruptedException/ExecutionException/TimeoutException) is attached.

Source

Thrown at pulsar-metadata/src/main/java/org/apache/pulsar/metadata/tableview/impl/MetadataStoreTableViewImpl.java:345

                });
    }

    /**
     * Note: this method only be called when a broker is starting, please call {@link #fillAsync(AtomicLong, boolean)}
     * for other use-case, otherwise, you may get a thread deadlock error.
     */
    private void fill() throws MetadataStoreException {
        AtomicLong loadedCounter = new AtomicLong();
        long maxWaitTime = Math.min(timeoutInMillis, FILL_TIMEOUT_IN_MILLIS);
        try {
            fillAsync(loadedCounter, false).get(maxWaitTime, TimeUnit.MILLISECONDS);
            log.info().attr("name", name).attr("size", loadedCounter.get()).log("Completed filling existing items");
        } catch (InterruptedException | ExecutionException | TimeoutException e) {
            String err = name + " failed to fill existing items in "
                    + TimeUnit.MILLISECONDS.toSeconds(maxWaitTime) + " secs. Filled count:"
                    + loadedCounter.get();
            log.error(err);
            throw new MetadataStoreException(err, FutureUtil.unwrapCompletionException(e));
        }
    }

    private CompletableFuture<Void> handleExistingLeafs(String rootPath, String path, @Nullable AtomicLong count,
                                                        boolean printDetails) {
        return store.getChildren(path).thenCompose(children -> {
            if (children.isEmpty()) {
                // Skip root path.
                if (rootPath.equals(path)) {
                    return CompletableFuture.completedFuture(null);
                }
                // Leaf node.
                if (count != null) {
                    count.incrementAndGet();
                }
                if (printDetails) {
                    log.info().attr("path", path).log("Handling existing leaf");
                }

View on GitHub (pinned to 820761864e)

Solutions

Increase maxWaitTime on the TableViewBuilder (tableview builder maxWaitTime option) to allow full initial load.
Check metadata store health and network latency between client and store.
Reduce the number of keys under the tableview root path, or use a narrower rootPath.
Inspect the wrapped cause (TimeoutException vs ExecutionException) to distinguish slowness from hard store failures; retry startup after the store recovers.

Example fix

// before
client.newTableViewBuilder(Schema.STRING)
    .topic(...)
    .maxWaitTime(30_000)
// after
client.newTableViewBuilder(Schema.STRING)
    .topic(...)
    .maxWaitTime(300_000)

Defensive patterns

Strategy: retry

Validate before calling

// estimate needed wait: e.g. sample child count first
Long childCount = store.getChildren(rootPath).get(30, TimeUnit.SECONDS);
long budgetMs = Math.max(60_000, childCount * 100); // heuristic
builder.maxWaitTime(budgetMs);

Try / catch

try {
    tableView = builder.createAsync().get(maxWaitTime, TimeUnit.MILLISECONDS);
} catch (ExecutionException e) {
    if (FutureUtil.unwrapCompletionException(e) instanceof TimeoutException) {
        // retry with larger maxWaitTime after checking store health
    }
}

Prevention

Set maxWaitTime proportional to expected key count under the tableview root.
Monitor metadata store latency and alert before timeouts trigger.
Avoid starting many tableviews simultaneously against a cold store.
Check the wrapped cause to distinguish timeout from store failure.

When it happens

Trigger: Starting a TableView where the backing metadata store has many children under the root path, or the metadata store (e.g. ZooKeeper) is slow/unavailable so the getChildren futures don't finish before maxWaitTime elapses.

Common situations: Large existing datasets in a tableview namespace, metadata store latency or network issues, or setting TableViewBuilder.createAsync maxWaitTime too low for the number of keys.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/a5c5a035259a6a03. Report an issue: GitHub.