{"record":{"id":"2fcba2d6b402b37e","repo":"apache/hadoop","slug":"incompatible-clusterid-for-journal-namenode-ha","errorCode":null,"errorMessage":"Incompatible clusterID for journal {}: NameNode has clusterId '{}' but storage has clusterId '{}'","messagePattern":"Incompatible clusterID for journal (.+?): NameNode has clusterId '(.+?)' but storage has clusterId '(.+?)'","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JNStorage.java","lineNumber":275,"sourceCode":"        && state != StorageState.NON_EXISTENT\n        && state != StorageState.NOT_FORMATTED;\n    if (state == StorageState.NORMAL && startOpt != StartupOption.ROLLBACK) {\n      readProperties(sd);\n    } else if (needRecover) {\n      sd.doRecover(state);\n    }\n  }\n\n  void checkConsistentNamespace(NamespaceInfo nsInfo)\n      throws IOException {\n    if (nsInfo.getNamespaceID() != getNamespaceID()) {\n      throw new IOException(\"Incompatible namespaceID for journal \" +\n          this.sd + \": NameNode has nsId \" + nsInfo.getNamespaceID() +\n          \" but storage has nsId \" + getNamespaceID());\n    }\n    \n    if (!nsInfo.getClusterID().equals(getClusterID())) {\n      throw new IOException(\"Incompatible clusterID for journal \" +\n          this.sd + \": NameNode has clusterId '\" + nsInfo.getClusterID() +\n          \"' but storage has clusterId '\" + getClusterID() + \"'\");\n      \n    }\n  }\n\n  public void close() throws IOException {\n    LOG.info(\"Closing journal storage for {}\", sd);\n    unlockAll();\n  }\n\n  public boolean isFormatted() {\n    return state == StorageState.NORMAL;\n  }\n}\n","sourceCodeStart":257,"sourceCodeEnd":291,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JNStorage.java#L257-L291","documentation":"The clusterID half of JNStorage.checkConsistentNamespace: during newEpoch the JournalNode's stored clusterID does not equal the clusterID in the NameNode's NamespaceInfo. The JN belongs to a different Hadoop cluster than the NN is claiming to be — a configuration or stale-format problem, never a transient fault.","triggerScenarios":"newEpoch from an NN whose clusterID differs from the JN's stored value: NN reformatted with a new auto-generated clusterID while JN dirs kept the old one; shared edits config pointing at a JN from another cluster; JN dir copied across environments (prod into test).","commonSituations":"Reformat of NN without re-initializing shared edits; environment cloning that copies JN data; editing clusterID by hand in one place only; multiple clusters sharing JN hosts with overlapping journal ids.","solutions":["If the NN format is the keeper: stop everything, wipe the JN journal dirs for this journal id, restart JNs, run 'hdfs namenode -initializeSharedEdits' to stamp the NN's clusterID/namespaceID on all JNs.","If the JN's cluster is the keeper (wrong NN pointing at it): fix the NN's clusterID or repoint dfs.namenode.shared.edits.dir to the correct JournalNodes.","Cross-check clusterID in the NN's and every JN's current/VERSION; they must all match.","Guard against recurrence: make reformatting a documented procedure that always includes JN dir cleanup."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Preflight: clusterID must be identical on NN and every JN\nString nnClusterId = NameNode.getNamespaceInfo().getClusterID();\nfor (URI jn : sharedEditsUris) {\n  String jnClusterId = readJnVersionFile(jn).clusterID;\n  if (!nnClusterId.equals(jnClusterId)) throw new IllegalStateException(\n      \"JN \" + jn + \" clusterId \" + jnClusterId + \" != NN \" + nnClusterId);\n}","typeGuard":"static boolean isClusterIdMismatch(IOException ioe) {\n  return ioe.getMessage() != null && ioe.getMessage().startsWith(\"Incompatible clusterID\");\n}","tryCatchPattern":"try {\n  journalRpcServerCall();\n} catch (IOException ioe) {\n  if (isClusterIdMismatch(ioe)) {\n    // identity conflict — retrying is pointless; reconcile formats/config\n    haltAndAlertAdmin(ioe);\n  } else {\n    throw ioe;\n  }\n}","preventionTips":["Set a stable clusterID by hand (or reuse the existing one) when reformatting part of a cluster.","Never clone JN data directories across clusters/environments.","Verify clusterID in current/VERSION on NN and JNs as a standing pre-flight check in deploy tooling."],"tags":["qjm","journalnode","cluster-id-mismatch","format","configuration"],"backgroundTag":"cluster-identity-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}