{"record":{"id":"f8dc9b2414cd6158","repo":"apache/pulsar","slug":"error-while-parsing-zk-protobuf-binary-data","errorCode":null,"errorMessage":"Error while parsing ZK protobuf binary data","messagePattern":"Error while parsing ZK protobuf binary data","errorType":"exception","errorClass":"ReplicationException.UnavailableException","httpStatus":null,"severity":"error","filePath":"pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java","lineNumber":954,"sourceCode":"        log.debug(\"getCheckAllLedgersCTime\");\n        try {\n            Optional<GetResult> optRes = store.get(checkAllLedgersCtimePath).get(BLOCKING_CALL_TIMEOUT, MILLISECONDS);\n            if (!optRes.isPresent()) {\n                log.warn(\"checkAllLedgersCtimeZnode is not yet available\");\n                return -1;\n            }\n            byte[] data = optRes.get().getValue();\n            CheckAllLedgersFormat checkAllLedgersFormat = new CheckAllLedgersFormat();\n            checkAllLedgersFormat.parseFrom(data);\n            return checkAllLedgersFormat.hasCheckAllLedgersCTime() ? checkAllLedgersFormat.getCheckAllLedgersCTime()\n                    : -1;\n        } catch (ExecutionException | TimeoutException ee) {\n            throw new ReplicationException.UnavailableException(\"Error contacting zookeeper\", ee);\n        } catch (InterruptedException ie) {\n            Thread.currentThread().interrupt();\n            throw new ReplicationException.UnavailableException(\"Interrupted while contacting zookeeper\", ie);\n        } catch (RuntimeException ipbe) {\n            throw new ReplicationException.UnavailableException(\"Error while parsing ZK protobuf binary data\", ipbe);\n        }\n    }\n\n    @Override\n    public void setPlacementPolicyCheckCTime(long placementPolicyCheckCTime) throws\n            ReplicationException.UnavailableException {\n        log.debug(\"setPlacementPolicyCheckCTime\");\n        try {\n            PlacementPolicyCheckFormat builder = new PlacementPolicyCheckFormat();\n            builder.setPlacementPolicyCheckCTime(placementPolicyCheckCTime);\n            byte[] placementPolicyCheckFormatByteArray = builder.toByteArray();\n            store.put(placementPolicyCheckCtimePath, placementPolicyCheckFormatByteArray, Optional.empty())\n                    .get(BLOCKING_CALL_TIMEOUT, MILLISECONDS);\n        } catch (ExecutionException | TimeoutException ke) {\n            throw new ReplicationException.UnavailableException(\"Error contacting zookeeper\", ke);\n        } catch (InterruptedException ie) {\n            Thread.currentThread().interrupt();\n            throw new ReplicationException.UnavailableException(\"Interrupted while contacting zookeeper\", ie);","sourceCodeStart":936,"sourceCodeEnd":972,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-metadata/src/main/java/org/apache/pulsar/metadata/bookkeeper/PulsarLedgerUnderreplicationManager.java#L936-L972","documentation":"Thrown by PulsarLedgerUnderreplicationManager.getCheckAllLedgersCTime when the bytes stored under the checkAllLedgers ctime znode cannot be parsed as a CheckAllLedgersFormat protobuf (a RuntimeException from parseFrom). It indicates corrupted or unexpected data at the well-known path, not a connectivity problem. Wrapped as ReplicationException.UnavailableException.","triggerScenarios":"Calling getCheckAllLedgersCTime when the znode contains non-protobuf bytes — data written by an incompatible or older version, manually edited/created znodes, truncated writes, or data created by a different tooling path.","commonSituations":"Pulsar/BookKeeper version migration where the stored format changed; operators manually creating or restoring znodes from a backup; corruption after an aborted write or bad ZooKeeper snapshot restore.","solutions":["Inspect the znode data (e.g. zookeeper-cli / getPersistedData) and confirm it is a serialized CheckAllLedgersFormat.","Delete the corrupted checkAllLedgers ctime znode; the code treats a missing node as -1 and it will be recreated on the next setCheckAllLedgersCTime.","Check for version mismatch between the component that wrote the znode and the one reading it; upgrade/downgrade consistently.","Restore the node from a healthy ZooKeeper snapshot taken before the corruption."],"exampleFix":"// before\nlong ctime = urManager.getCheckAllLedgersCTime(); // throws on corrupt data\n// after\nlong ctime;\ntry {\n    ctime = urManager.getCheckAllLedgersCTime();\n} catch (ReplicationException.UnavailableException e) {\n    log.warn(\"Corrupt checkAllLedgers ctime data, resetting\", e);\n    urManager.setCheckAllLedgersCTime(System.currentTimeMillis()); // recreate znode\n    ctime = urManager.getCheckAllLedgersCTime();\n}","handlingStrategy":"fallback","validationCode":"// Optional: verify node content sanity via store API before parse-sensitive read\nbyte[] data = metadataStore.get(checkAllLedgersPath).get().orElse(null);\nboolean looksValid = data != null && data.length > 0;\nif (!looksValid) {\n    log.warn(\"checkAllLedgers ctime node missing/empty; will be recreated\");\n}","typeGuard":null,"tryCatchPattern":"long ctime;\ntry {\n    ctime = urManager.getCheckAllLedgersCTime();\n} catch (ReplicationException.UnavailableException e) {\n    if (e.getCause() instanceof RuntimeException) {\n        // corrupt data: fall back to sentinel and reset\n        ctime = -1;\n        urManager.setCheckAllLedgersCTime(System.currentTimeMillis());\n    } else {\n        throw e; // connectivity/timeout: different remediation\n    }\n}","preventionTips":["Never hand-edit or create internal bookkeeping znodes manually.","Keep broker and BookKeeper versions aligned across upgrades.","Validate ZooKeeper snapshot restores before pointing a cluster at them.","Alert on parse failures so corruption is detected at first occurrence."],"tags":["zookeeper","protobuf","data-corruption","bookkeeper","parsing"],"backgroundTag":"corrupt-protobuf-data","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}