apache/pulsar · critical · RuntimeException

Failed to initialize Metadata Manager

Error message

Failed to initialize Metadata Manager

What it means

FunctionMetaDataManager.initialize replays all messages from the function metadata topic before the worker serves requests. Any failure (reader creation, read errors, processing exceptions) is logged and rethrown as RuntimeException 'Failed to initialize Metadata Manager', aborting worker startup.

Source

Thrown at pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/FunctionMetaDataManager.java:113

    /*
     * Public methods. Please use these methods if references FunctionMetaManager from an external class
     */

    /**
     * Initializes the FunctionMetaDataManager.
     * We create a new reader
     */
    public synchronized void initialize() {
        try (Reader<byte[]> reader = FunctionMetaDataTopicTailer.createReader(
                workerConfig, pulsarClient.newReader(), MessageId.earliest)) {
            // read all existing messages
            while (reader.hasMessageAvailable()) {
                processMetaDataTopicMessage(reader.readNext());
            }
            this.isInitialized.complete(null);
        } catch (Exception e) {
            log.error().exception(e).log("Failed to initialize meta data store");
            throw new RuntimeException("Failed to initialize Metadata Manager", e);
        }
        log.info("FunctionMetaData Manager initialization complete");
    }

    // Starts the tailer if we are in non-leader mode
    public synchronized void start() {
        if (exclusiveLeaderProducer == null) {
            try {
                // This means that we are in non-leader mode. start function metadata tailer
                initializeTailer();
            } catch (PulsarClientException e) {
                throw new RuntimeException("Could not start MetaData topic tailer", e);
            }
        }
    }

    @Override
    public void close() throws Exception {

View on GitHub (pinned to 820761864e)

Solutions

  1. Check connectivity to the Pulsar broker/BookKeeper and ensure the functions metadata topic exists and is readable
  2. Fix topic authorization for the worker's role on the metadata topic
  3. Inspect the metadata topic for a poison message; skip/repair it, or reinitialize the functions state

Example fix

// startup log shows cause 'Failed to read from metadata topic'
// fix: grant worker role read access
pulsar-admin namespaces grant-permissions public/functions --role functions-worker --actions read,produce
Defensive patterns

Strategy: try-catch

Validate before calling

// before starting worker
pulsarAdmin.namespaces().getPermissions("public/functions"); // verify topic access
PulsarAdmin admin = ...;
admin.topics().getStats("persistent://public/functions/metadata"); // topic exists/readable

Try / catch

try {
    worker.start();
} catch (RuntimeException e) {
    if (e.getMessage().contains("Failed to initialize Metadata Manager")) {
        // check broker health / metadata topic permissions, then restart worker
        log.error("metadata manager init failed", e.getCause());
    } else { throw e; }
}

Prevention

When it happens

Trigger: Worker startup when the metadata topic (persistent://public/functions/metadata) cannot be read: broker unreachable, topic permissions issues, corrupted messages, or processMetaDataTopicMessage throwing on a bad record.

Common situations: BookKeeper/broker outage during worker start; ACLs preventing the worker from reading the metadata topic; an old/poison message in the metadata topic failing deserialization or validation.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/79e7878ae65fa5d5. Report an issue: GitHub.