{"record":{"id":"e828df0b2d57c089","repo":"apache/hadoop","slug":"incompatible-namespaceids-in-namenode-namespac-e828df","errorCode":null,"errorMessage":"Incompatible namespaceIDs in {}: namenode namespaceID = {}; datanode namespaceID = {}","messagePattern":"Incompatible namespaceIDs in (.+?): namenode namespaceID = (.+?); datanode namespaceID = (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java","lineNumber":741,"sourceCode":"      createStorageID(sd, layoutVersion, conf);\n      return false; // regular start up for PROVIDED storage directories\n    }\n    if (startOpt == StartupOption.ROLLBACK) {\n      doRollback(sd, nsInfo); // rollback if applicable\n    }\n    readProperties(sd);\n    checkVersionUpgradable(this.layoutVersion);\n    assert this.layoutVersion >=\n        DataNodeLayoutVersion.getCurrentLayoutVersion() :\n        \"Future version is not allowed\";\n    \n    boolean federationSupported = \n      DataNodeLayoutVersion.supports(\n          LayoutVersion.Feature.FEDERATION, layoutVersion);\n    // For pre-federation version - validate the namespaceID\n    if (!federationSupported &&\n        getNamespaceID() != nsInfo.getNamespaceID()) {\n      throw new IOException(\"Incompatible namespaceIDs in \"\n          + sd.getRoot().getCanonicalPath() + \": namenode namespaceID = \"\n          + nsInfo.getNamespaceID() + \"; datanode namespaceID = \"\n          + getNamespaceID());\n    }\n    \n    // For version that supports federation, validate clusterID\n    if (federationSupported\n        && !getClusterID().equals(nsInfo.getClusterID())) {\n      throw new IOException(\"Incompatible clusterIDs in \"\n          + sd.getRoot().getCanonicalPath() + \": namenode clusterID = \"\n          + nsInfo.getClusterID() + \"; datanode clusterID = \" + getClusterID());\n    }\n\n    // regular start up.\n    if (this.layoutVersion == DataNodeLayoutVersion.getCurrentLayoutVersion()) {\n      createStorageID(sd, layoutVersion, conf);\n      return false; // need to write properties\n    }","sourceCodeStart":723,"sourceCodeEnd":759,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java#L723-L759","documentation":"For pre-federation storage layouts, DataStorage compares the namespaceID stored in the directory's VERSION with the NamespaceInfo sent by the NameNode at registration. A mismatch means the datanode storage belongs to a different (pre-federation) namespace, and that directory fails to load with this IOException.","triggerScenarios":"Pre-federation DN storage connected to a re-formatted or different NameNode: nsInfo.getNamespaceID() != stored namespaceID when the stored layout version does not support the FEDERATION feature.","commonSituations":"Very old clusters (pre-0.22 layouts) after an NN re-format; test environments reusing legacy dirs against fresh NNs; historical upgrade paths skipping the documented procedure.","solutions":["If the data is disposable, clear/reformat the datanode storage for that namespace and re-register","Point the datanode at the NameNode whose namespaceID matches the stored one","For genuine old-cluster upgrades, follow the documented pre-federation upgrade procedure instead of a plain restart"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (String.valueOf(e.getMessage()).contains(\"Incompatible namespaceIDs\")) {\n    // pre-federation storage vs different NN namespace:\n    // point DN at the right NN, or reformat/clear the storage deliberately\n  } else { throw e; }\n}","preventionTips":["On legacy clusters, always clear DN storage in the same maintenance step as an NN re-format","Pin test clusters to a fixed namespaceID so accidental re-formats keep IDs stable","Verify the NN's namespaceID (its VERSION file) matches DN storage before pointing DNs at it"],"tags":["hadoop","hdfs","datanode","storage","namespaceid","mismatch","pre-federation"],"backgroundTag":"namespace-id-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}