{"record":{"id":"c23ff36d962c4cef","repo":"apache/pulsar","slug":"error-while-parsing-zk-data-of-lock","errorCode":null,"errorMessage":"Error while parsing ZK data of lock","messagePattern":"Error while parsing ZK data of lock","errorType":"exception","errorClass":"ReplicationException.UnavailableException","httpStatus":null,"severity":"error","filePath":"pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java","lineNumber":912,"sourceCode":"                // this is ok.\n                return null;\n            }\n\n            byte[] lockData = optRes.get().getValue();\n            LockDataFormat lock = new LockDataFormat();\n            lock.parseFromTextFormat(lockData);\n            return lock.getBookieId();\n        } catch (ExecutionException | TimeoutException e) {\n            log.error().exception(e).log(\"Error while getting ReplicationWorkerId rereplicating Ledger\");\n            throw new ReplicationException.UnavailableException(\n                    \"Error while getting ReplicationWorkerId rereplicating Ledger\", e);\n        } catch (InterruptedException e) {\n            log.error().exception(e).log(\"Got interrupted while getting ReplicationWorkerId rereplicating Ledger\");\n            Thread.currentThread().interrupt();\n            throw new ReplicationException.UnavailableException(\"Interrupted while contacting zookeeper\", e);\n        } catch (RuntimeException e) {\n            log.error().exception(e).log(\"Error while parsing ZK data of lock\");\n            throw new ReplicationException.UnavailableException(\"Error while parsing ZK data of lock\", e);\n        }\n    }\n\n    @Override\n    public void setCheckAllLedgersCTime(long checkAllLedgersCTime) throws ReplicationException.UnavailableException {\n        log.debug(\"setCheckAllLedgersCTime\");\n        try {\n            CheckAllLedgersFormat builder = new CheckAllLedgersFormat();\n            builder.setCheckAllLedgersCTime(checkAllLedgersCTime);\n            byte[] checkAllLedgersFormatByteArray = builder.toByteArray();\n\n            store.put(checkAllLedgersCtimePath, checkAllLedgersFormatByteArray, Optional.empty())\n                    .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS);\n        } catch (ExecutionException | TimeoutException ee) {\n            throw new ReplicationException.UnavailableException(\"Error contacting zookeeper\", ee);\n        } catch (InterruptedException ie) {\n            Thread.currentThread().interrupt();\n            throw new ReplicationException.UnavailableException(\"Interrupted while contacting zookeeper\", ie);","sourceCodeStart":894,"sourceCodeEnd":930,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java#L894-L930","documentation":"getReplicationWorkerIdRereplicatingLedger(long) parses the lock znode payload with LockDataFormat.parseFromTextFormat(). Any RuntimeException during parsing (malformed/unexpected bytes, legacy or hand-edited znode data, format drift between BookKeeper versions) is wrapped as ReplicationException.UnavailableException('Error while parsing ZK data of lock'). Unlike the other cases here, ZooKeeper is healthy — the stored lock data itself is unreadable.","triggerScenarios":"Calling getReplicationWorkerIdRereplicatingLedger(ledgerId) when the lock znode's value is not valid LockDataFormat text (corrupt entry, data written by an incompatible BookKeeper version, manual znode edit, NPE on empty bookieId field).","commonSituations":"Upgrading BookKeeper between lock-data format revisions leaving stale locks; a crashed ReplicationWorker leaving a truncated/partial lock payload; operators manually deleting/rewriting underreplication znodes during incident response.","solutions":["Dump the znode content (zkCli get <urLockPath>/<ledgerId>) and compare against the LockDataFormat schema for your BookKeeper version","Delete the stale/corrupt lock znode so the ledger re-enters the underreplication flow and a worker re-acquires it cleanly","After version upgrades, verify lock format compatibility before resuming the ReplicationWorker","If frequent, capture the raw payload in the log (the cause carries the parse exception) and report a format mismatch to the BookKeeper/Pulsar project"],"exampleFix":"// before\nString owner = urManager.getReplicationWorkerIdRereplicatingLedger(ledgerId); // dies on corrupt payload\n\n// after\nString owner;\ntry {\n    owner = urManager.getReplicationWorkerIdRereplicatingLedger(ledgerId);\n} catch (ReplicationException.UnavailableException e) {\n    if (e.getCause() instanceof RuntimeException) {\n        log.warn(\"corrupt lock data for ledger {} — releasing lock\", ledgerId, e);\n        urManager.releaseUnderreplicatedLedger(ledgerId);\n        owner = null;\n    } else {\n        throw e;\n    }\n}","handlingStrategy":"validation","validationCode":"// inspect the lock payload before relying on it\ntry {\n    String owner = urManager.getReplicationWorkerIdRereplicatingLedger(ledgerId);\n    if (owner == null || owner.isEmpty()) {\n        log.warn(\"ledger {} lock has no bookie id; releasing\", ledgerId);\n        urManager.releaseUnderreplicatedLedger(ledgerId);\n    }\n} catch (ReplicationException.UnavailableException e) {\n    if (e.getCause() instanceof RuntimeException) {\n        log.warn(\"corrupt lock payload for ledger {}\", ledgerId, e);\n        urManager.releaseUnderreplicatedLedger(ledgerId);\n    } else {\n        throw e;\n    }\n}","typeGuard":"static boolean isParseCorruption(ReplicationException.UnavailableException e) {\n    return e.getCause() instanceof RuntimeException\n        && !(e.getCause() instanceof IllegalStateException);\n}","tryCatchPattern":"try {\n    urManager.getReplicationWorkerIdRereplicatingLedger(ledgerId);\n} catch (ReplicationException.UnavailableException e) {\n    if (isParseCorruption(e)) {\n        urManager.releaseUnderreplicatedLedger(ledgerId); // let a worker re-lock cleanly\n    } else {\n        throw e;\n    }\n}","preventionTips":["Do not hand-edit underreplication znodes; use the admin API","Check BookKeeper/Pulsar lock-format compatibility notes before upgrades","Monitor for RuntimeException causes in UnavailableException to detect corrupt locks early","After forced lock cleanup, verify the ledger re-enters the underreplication flow"],"tags":["data-corruption","parse-error","zookeeper","ledger-lock","compatibility"],"backgroundTag":"lock-data-corrupt","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}