apache/pulsar · error · ReplicationException.UnavailableException

Exception while stopping auto ledger re-replication

Error message

Exception while stopping auto ledger re-replication

What it means

Thrown by disableLedgerReplication() when writing the replication-disable marker (store.put on replicationDisablePath) fails with ExecutionException or TimeoutException. BookKeeper cannot stop auto re-replication because the marker could not be persisted; wrapped as ReplicationException.UnavailableException.

Source

Thrown at pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java:722

        } catch (TimeoutException ex) {
            throw new ReplicationException.UnavailableException("Error contacting metadata store", ex);
        } catch (InterruptedException ie) {
            Thread.currentThread().interrupt();
            throw new ReplicationException.UnavailableException("Interrupted while connecting metadata store", ie);
        }
    }

    @Override
    public void disableLedgerReplication()
            throws ReplicationException.UnavailableException {
        log.debug("disableLedgerReplication()");
        try {
            store.put(replicationDisablePath, "".getBytes(UTF_8), Optional.of(-1L))
                    .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS);
            log.info("Auto ledger re-replication is disabled!");
        } catch (ExecutionException | TimeoutException ee) {
            log.error().exception(ee).log("Exception while stopping auto ledger re-replication");
            throw new ReplicationException.UnavailableException(
                    "Exception while stopping auto ledger re-replication", ee);
        } catch (InterruptedException ie) {
            Thread.currentThread().interrupt();
            throw new ReplicationException.UnavailableException(
                    "Interrupted while stopping auto ledger re-replication", ie);
        }
    }

    @Override
    public void enableLedgerReplication()
            throws ReplicationException.UnavailableException {
        log.debug("enableLedgerReplication()");
        try {
            store.delete(replicationDisablePath, Optional.empty())
                    .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS);
            log.info("Resuming automatic ledger re-replication");
        } catch (ExecutionException | TimeoutException ee) {
            log.error().exception(ee).log("Exception while resuming ledger replication");

View on GitHub (pinned to 820761864e)

Solutions

Verify metadata store availability and ACL write permission on the replication disable path, then retry the disable.
Check the cause: auth/noauth or readonly errors require fixing credentials/client config; timeouts require network/quorum fixes.
Retry disableLedgerReplication() after recovery — the put is idempotent.
If during planned maintenance, ensure the metadata store is healthy before starting the procedure.

Example fix

// before
manager.disableLedgerReplication();
// after
try {
    manager.disableLedgerReplication();
} catch (ReplicationException.UnavailableException e) {
    log.warn("Could not persist replication-disable marker; retrying", e);
    retryWithBackoff(() -> manager.disableLedgerReplication());
}

Defensive patterns

Strategy: retry

Validate before calling

// Java: pre-check store writability before disabling replication
store.exists(replicationDisablePath).get(5, TimeUnit.SECONDS);
// plus: confirm ACLs allow put and server is not read-only

Type guard

static boolean storeFailure(ReplicationException.UnavailableException e) {
    Throwable c = e.getCause();
    return c instanceof ExecutionException || c instanceof TimeoutException;
}

Try / catch

try {
    manager.disableLedgerReplication();
} catch (ReplicationException.UnavailableException e) {
    backoffAndRetry(() -> manager.disableLedgerReplication()); // put is idempotent
}

Prevention

Grant write ACLs on the replication-disable path to the bookie identity.
Confirm store health before maintenance windows that require disabling re-replication.
Retry idempotent puts with bounded backoff.
Alert on read-only/observer misconfiguration of the metadata store client.

When it happens

Trigger: Calling disableLedgerReplication() (e.g. via autorecovery admin or bookie health check) while the metadata store is down, the put is rejected (no auth, read-only server, connection loss), or the future times out.

Common situations: ZooKeeper ensemble outage during a maintenance window where an operator disables re-replication, insufficient ACLs on the replication disable node, or client connected to a read-only ZooKeeper observer.

Related errors

AI-assisted analysis of apache/pulsar@820761864e (2026-09-06). Data as JSON: /api/errors/bf3896c19371946c. Report an issue: GitHub.