{"record":{"id":"8ae599d09c652a25","repo":"apache/pulsar","slug":"exception-while-resuming-auto-ledger-re-replicatio","errorCode":null,"errorMessage":"Exception while resuming auto ledger re-replication","messagePattern":"Exception while resuming auto ledger re-replication","errorType":"exception","errorClass":"ReplicationException.UnavailableException","httpStatus":null,"severity":"error","filePath":"pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java","lineNumber":741,"sourceCode":"                    \"Exception while stopping auto ledger re-replication\", ee);\n        } catch (InterruptedException ie) {\n            Thread.currentThread().interrupt();\n            throw new ReplicationException.UnavailableException(\n                    \"Interrupted while stopping auto ledger re-replication\", ie);\n        }\n    }\n\n    @Override\n    public void enableLedgerReplication()\n            throws ReplicationException.UnavailableException {\n        log.debug(\"enableLedgerReplication()\");\n        try {\n            store.delete(replicationDisablePath, Optional.empty())\n                    .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS);\n            log.info(\"Resuming automatic ledger re-replication\");\n        } catch (ExecutionException | TimeoutException ee) {\n            log.error().exception(ee).log(\"Exception while resuming ledger replication\");\n            throw new ReplicationException.UnavailableException(\n                    \"Exception while resuming auto ledger re-replication\", ee);\n        } catch (InterruptedException ie) {\n            Thread.currentThread().interrupt();\n            throw new ReplicationException.UnavailableException(\n                    \"Interrupted while resuming auto ledger re-replication\", ie);\n        }\n    }\n\n    @Override\n    public boolean isLedgerReplicationEnabled()\n            throws ReplicationException.UnavailableException {\n        log.debug(\"isLedgerReplicationEnabled()\");\n        try {\n            return !store.exists(replicationDisablePath)\n                    .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS);\n        } catch (ExecutionException | TimeoutException ee) {\n            log.error().exception(ee).log(\"Error while checking the state of ledger re-replication\");\n            throw new ReplicationException.UnavailableException(","sourceCodeStart":723,"sourceCodeEnd":759,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java#L723-L759","documentation":"Thrown by PulsarLedgerUnderreplicationManager.enableLedgerReplication() when the blocking metadata-store delete of the replication-disable node (replicationDisablePath) fails or times out. BookKeeper's underreplication manager uses this node to disable auto ledger re-replication; enabling means deleting it. If the underlying MetadataStore operation completes exceptionally (ExecutionException) or does not finish within BLOCKING_CALL_TIMEOUT (TimeoutException), the manager wraps it in ReplicationException.UnavailableException.","triggerScenarios":"Calling enableLedgerReplication() (or its Admin API equivalent) when the metadata store (ZooKeeper/etcd) is down, unreachable, session expired, slow, or the delete exceeds BLOCKING_CALL_TIMEOUT.","commonSituations":"ZooKeeper quorum lost or overloaded during a cluster incident; network partition between broker and metadata store; long GC pauses on ZooKeeper making the delete exceed the blocking timeout; calling the admin API while the metadata service is restarting.","solutions":["Check metadata-store connectivity and health (zookeeper quorum, network, TLS) from the broker.","Retry enableLedgerReplication() after the store recovers; the delete is idempotent.","Inspect the cause (getCause()) for ExecutionException to find the exact store failure (session expired, no quorum, etc.).","Increase timeout pressure margin (e.g. fix ZooKeeper latency) if timeouts recur under load."],"exampleFix":"// before\nauditor.enableLedgerReplication(); // throws on transient ZK blip\n// after\ntry {\n    auditor.enableLedgerReplication();\n} catch (ReplicationException.UnavailableException e) {\n    log.warn(\"Metadata store unavailable, will retry\", e);\n    // schedule retry after store health check\n}","handlingStrategy":"retry","validationCode":"// verify store reachable before enabling\nif (!store.exists(replicationDisablePath).get(5, TimeUnit.SECONDS) != null) {\n    throw new IllegalStateException(\"Metadata store unreachable\");\n}","typeGuard":"boolean isMetadataStoreHealthy(MetadataStore store) {\n    try {\n        store.exists(\"/health-check-probe\").get(5, TimeUnit.SECONDS);\n        return true;\n    } catch (Exception e) {\n        return false;\n    }\n}","tryCatchPattern":"try {\n    urManager.enableLedgerReplication();\n} catch (ReplicationException.UnavailableException e) {\n    Throwable cause = e.getCause();\n    if (cause instanceof TimeoutException) {\n        // retry with backoff\n    } else {\n        // alert on store connectivity\n    }\n}","preventionTips":["Monitor metadata store health before issuing admin replication operations.","Retry idempotent enable/delete operations with exponential backoff.","Alert on ZooKeeper session expiry and quorum loss so operations are not attempted during outages."],"tags":["metadata-store","zookeeper","timeout","bookkeeper-replication"],"backgroundTag":"metadata-store-unavailable","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}