{"record":{"id":"2f65fd0739f3fa0e","repo":"apache/hadoop","slug":"cannot-load-data-from-zookeeper-itwas-written-wit","errorCode":null,"errorMessage":"Cannot load data from ZooKeeper; itwas written with a newer version","messagePattern":"Cannot load data from ZooKeeper; itwas written with a newer version","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java","lineNumber":313,"sourceCode":"    bb.putLong(nextRolloverDate);\n    return bb.array();\n  }\n\n  /**\n   * Pulls data from ZooKeeper.  If isInit is false, it will only parse the\n   * next secret and version.  If isInit is true, it will also parse the current\n   * and previous secrets, and the next rollover date; it will also init the\n   * secrets.  Hence, isInit should only be true on startup.\n   * @param isInit  see description above\n   */\n  private synchronized void pullFromZK(boolean isInit) {\n    try {\n      Stat stat = new Stat();\n      byte[] bytes = client.getData().storingStatIn(stat).forPath(path);\n      ByteBuffer bb = ByteBuffer.wrap(bytes);\n      int dataVersion = bb.getInt();\n      if (dataVersion > DATA_VERSION) {\n        throw new IllegalStateException(\"Cannot load data from ZooKeeper; it\"\n                + \"was written with a newer version\");\n      }\n      int nextSecretLength = bb.getInt();\n      byte[] nextSecret = new byte[nextSecretLength];\n      bb.get(nextSecret);\n      this.nextSecret = nextSecret;\n      zkVersion = stat.getVersion();\n      if (isInit) {\n        int currentSecretLength = bb.getInt();\n        byte[] currentSecret = new byte[currentSecretLength];\n        bb.get(currentSecret);\n        int previousSecretLength = bb.getInt();\n        byte[] previousSecret = null;\n        if (previousSecretLength > 0) {\n          previousSecret = new byte[previousSecretLength];\n          bb.get(previousSecret);\n        }\n        super.initSecrets(currentSecret, previousSecret);","sourceCodeStart":295,"sourceCodeEnd":331,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java#L295-L331","documentation":"ZKSignerSecretProvider serializes secrets into a znode with a leading data-format version integer. On pullFromZK(), if the stored version is greater than the DATA_VERSION this JVM's hadoop-auth was built with, it throws IllegalStateException('Cannot load data from ZooKeeper; itwas written with a newer version') — a forward-compatibility guard, since older code cannot safely interpret newer payloads.","triggerScenarios":"A cluster node running an older Hadoop version connects to a ZooKeeper znode whose secret data was written by an upgraded (newer) node; mixed-version rolling upgrade while the zookeeper secret provider is active.","commonSituations":"Rolling upgrades where some RMS/UI nodes are upgraded first and write the new-format znode; a test/dev environment pointing at a shared ZooKeeper used by newer software.","solutions":["Complete the rolling upgrade so every node reading that znode runs the newer hadoop-auth version","Or, during upgrade windows, point nodes at a fresh znode path (signer.secret.provider.zookeeper.path) to start with current-format data, accepting re-authentication","Do not permanently run mixed versions against one secret znode"],"exampleFix":"# before: old node reads znode written by new node\nsigner.secret.provider.zookeeper.path=/hadoop-auth-secret\n\n# after (during upgrade): isolate old fleet on a fresh path\nsigner.secret.provider.zookeeper.path=/hadoop-auth-secret-v2","handlingStrategy":"fallback","validationCode":"// cannot pre-validate remotely stored bytes; after getData, provider checks version itself — mirror the check when reading custom ZK data\nif (dataVersion > DATA_VERSION) throw new IllegalStateException(\"newer znode data\");","typeGuard":null,"tryCatchPattern":"try { /* provider init reads znode */ } catch (IllegalStateException e) { /* mixed-version cluster: finish upgrade or switch to a fresh znode path */ }","preventionTips":["Avoid permanently mixed Hadoop versions sharing one secret znode","During rolling upgrades, plan a fresh znode path or complete the upgrade before re-enabling nodes","Keep dev/test clusters pointed at their own ZooKeeper namespace"],"tags":["hadoop-auth","zookeeper","version-mismatch","rolling-upgrade"],"backgroundTag":"data-version-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}