apache/pulsar · critical · IOException

Failed to initialize BookKeeper metadata

Error message

Failed to initialize BookKeeper metadata

What it means

PulsarClusterMetadataSetup.initializeCluster formats the BookKeeper metadata store (creating the /ledgers znode hierarchy) only if BookKeeperConstants.DEFAULT_ZK_LEDGERS_ROOT_PATH does not exist. If the path is absent and BookKeeperAdmin.format(bkConf, interactive=false, force=false) fails, this IOException is thrown, aborting cluster metadata initialization.

Source

Thrown at pulsar-broker/src/main/java/org/apache/pulsar/PulsarClusterMetadataSetup.java:321

                arguments.metadataStoreConfigPath,
                arguments.zkSessionTimeoutMillis);
        MetadataStoreExtended configStore = initConfigMetadataStore(arguments.configurationMetadataStore,
                arguments.configurationStoreConfigPath,
                arguments.zkSessionTimeoutMillis);
        try {

        final String metadataStoreUrlNoIdentifier = MetadataStoreFactoryImpl
                .removeIdentifierFromMetadataURL(arguments.metadataStoreUrl);
        // Format BookKeeper ledger storage metadata
        if (arguments.existingBkMetadataServiceUri == null && arguments.bookieMetadataServiceUri == null) {
            ServerConfiguration bkConf = new ServerConfiguration();
            bkConf.setListDelimiterHandler(new DisabledListDelimiterHandler());
            bkConf.setMetadataServiceUri("metadata-store:" + arguments.metadataStoreUrl);
            bkConf.setZkTimeout(arguments.zkSessionTimeoutMillis);
            // only format if /ledgers doesn't exist
            if (!localStore.exists(BookKeeperConstants.DEFAULT_ZK_LEDGERS_ROOT_PATH).get()
                && !BookKeeperAdmin.format(bkConf, false /* interactive */, false /* force */)) {
                throw new IOException("Failed to initialize BookKeeper metadata");
            }
        }

        if (localStore instanceof DualMetadataStore && configStore instanceof DualMetadataStore) {
            String uriStr;
            if (arguments.existingBkMetadataServiceUri != null) {
                uriStr = arguments.existingBkMetadataServiceUri;
            } else if (arguments.bookieMetadataServiceUri != null) {
                uriStr = arguments.bookieMetadataServiceUri;
            } else {
                uriStr = "zk+null://" + metadataStoreUrlNoIdentifier + BookKeeperConstants.DEFAULT_ZK_LEDGERS_ROOT_PATH;
            }

            // initial distributed log metadata
            initialDlogNamespaceMetadata(arguments.configurationMetadataStore, uriStr);

            ServiceURI bkMetadataServiceUri = ServiceURI.create(uriStr);
            // Format BookKeeper stream storage metadata

View on GitHub (pinned to 820761864e)

Solutions

1. Verify the metadata store (ZooKeeper) is reachable at the configured --zk-servers / metadata store URL
2. Inspect ZooKeeper for a partial layout (/admin and /ledgers); clean up stale state or use format with force via BookKeeperAdmin when safe
3. Re-run pulsar initialize-cluster-metadata after fixing connectivity
4. Check ZooKeeper ACLs/credentials allow creating the /ledgers node

Example fix

// before
bkConf.setMetadataServiceUri("metadata-store:zk:wrong-host:2181");
// after
bkConf.setMetadataServiceUri("metadata-store:zk:zk1:2181,zk2:2181,zk3:2181");

MetadataStore store = PulsarMetadataStore.create(metadataStoreUrl);
if (!store.exists("/ledgers").get()) {
    // verify connectivity before attempting format
    store.getChildren("/").get();
}

try {
    initializeCluster(arguments, configStore, localStore);
} catch (IOException e) {
    if (e.getMessage().contains("Failed to initialize BookKeeper metadata")) {
        throw new IllegalStateException("Check metadata store reachability and existing BK layout under /ledgers, then retry", e);
    }
    throw e;
}

When it happens

Trigger: Running 'pulsar initialize-cluster-metadata' against a metadata store where /ledgers doesn't exist and format fails — e.g. metadata service unreachable or refusing the format, ZK ACL/auth issues, or metadata-store URL misconfigured.

Common situations: ZooKeeper not running or wrong zkServers in the metadata store URL; partial previous initialization leaving inconsistent state; formatting while the BK metadata layout already exists but /ledgers is missing; connectivity/auth failures to the metadata store.

Related errors

• Timeout during update managedLedger's properties
• Error contacting with metadata store
• Error contacting metadata store
• Error while getting ReplicationWorkerId rereplicating Ledger
• Error while parsing ZK protobuf binary data

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/b6142c07d777f439. Report an issue: GitHub.