apache/pulsar · warning · ReplicationException.UnavailableException

Interrupted while contacting metadata store

Error message

Interrupted while contacting metadata store

What it means

Thrown by markLedgerReplicated when the thread waiting on the blocking metadata-store future is interrupted; the InterruptedException cause carries the interruption. The method re-asserts the interrupt flag (Thread.currentThread().interrupt()) before wrapping, so interruption is not swallowed. This is ReplicationException.UnavailableException, i.e. the operation did not complete because the caller stopped waiting.

Solutions

  1. Check whether the broker/auditor was shutting down at the time; if so this is expected and needs no fix.
  2. Avoid interrupting the thread running markLedgerReplicated; let in-flight rereplication finish before calling shutdownNow().
  3. Re-check the interrupt flag in your caller (it is preserved) and abort gracefully rather than retrying immediately.
  4. If interruptions come from your own executor, use awaitTermination with a generous timeout after shutdown() before shutdownNow().

Example fix

// before: swallowing the exception and losing the interrupt flag
catch (ReplicationException.UnavailableException e) { /* ignore */ }

// after: detect interruption and stop the loop
} catch (ReplicationException.UnavailableException e) {
    if (Thread.currentThread().isInterrupted()) {
        log.info("Interrupted during markLedgerReplicated, stopping rereplication loop");
        return;
    }
    throw e;
}
Defensive patterns

Strategy: try-catch

Validate before calling

// Java: skip the call when the thread is already interrupted
if (Thread.currentThread().isInterrupted()) {
    return; // don't start blocking metadata-store work while interrupted
}

Try / catch

try {
    manager.markLedgerReplicated(ledgerId);
} catch (ReplicationException.UnavailableException e) {
    if (Thread.currentThread().isInterrupted()) {
        Thread.currentThread().interrupt(); // preserve flag, stop work
        return;
    }
    throw e; // genuine store unavailability
}

Prevention

When it happens

Trigger: Calling markLedgerReplicated(ledgerId) while the calling thread (typically the auditor's scheduled executor or a shutdown hook) is interrupted, e.g. during broker shutdown, executor.shutdownNow(), or a task cancelled while blocked in .get(BLOCKING_CALL_TIMEOUT) on store.delete/store.get.

Common situations: Broker/auditor shutdown racing with ledger replication completion; BookKeeper client closing its callback executors; tests cancelling futures; thread pools being torn down while rereplication work is in flight.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/93ee5aba79d5f16e. Report an issue: GitHub.

Appendix: source

Thrown at pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java:485

                    }
                }
            }
        } catch (ExecutionException ee) {
            if (ee.getCause() instanceof MetadataStoreException.NotFoundException) {
                // this is ok
            } else if (ee.getCause() instanceof MetadataStoreException.BadVersionException) {
                // if this is the case, some has marked the ledger
                // for rereplication again. Leave the underreplicated
                // znode in place, so the ledger is checked.
            } else {
                log.error().exception(ee).log("Error deleting underreplicated ledger node");
                throw new ReplicationException.UnavailableException("Error contacting metadata store", ee);
            }
        } catch (TimeoutException ex) {
            throw new ReplicationException.UnavailableException("Error contacting metadata store", ex);
        } catch (InterruptedException ie) {
            Thread.currentThread().interrupt();
            throw new ReplicationException.UnavailableException("Interrupted while contacting metadata store", ie);
        } finally {
            releaseUnderreplicatedLedger(ledgerId);
        }
    }

    /**
     * Get a list of all the underreplicated ledgers which have been
     * marked for rereplication, filtered by the predicate on the replicas list.
     *
     * <p>Replicas list of an underreplicated ledger is the list of the bookies which are part of
     * the ensemble of this ledger and are currently unavailable/down.
     *
     * @param predicate filter to use while listing under replicated ledgers. 'null' if filtering is not required.
     * @return an iterator which returns underreplicated ledgers.
     */
    @Override
    public Iterator<UnderreplicatedLedger> listLedgersToRereplicate(final Predicate<List<String>> predicate) {
        final Queue<String> queue = new LinkedList<>();

View on GitHub (pinned to 820761864e)