{"record":{"id":"281efe1e7c405255","repo":"apache/hadoop","slug":"storage-directories-contain-multiple-layout-versio","errorCode":null,"errorMessage":"Storage directories contain multiple layout versions: {layoutVersions}","messagePattern":"Storage directories contain multiple layout versions: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java","lineNumber":1124,"sourceCode":"        continue;\n      }\n      readProperties(sd, startupOption); // sets layoutVersion\n      int lv = getLayoutVersion();\n      if (layoutVersion == null) {\n        layoutVersion = lv;\n      } else if (!layoutVersion.equals(lv)) {\n        multipleLV = true;\n      }\n      layoutVersions.append(\"(\").append(sd.getRoot()).append(\", \").append(lv)\n          .append(\") \");\n    }\n    \n    if (layoutVersion == null) {\n      throw new IOException(\"No storage directories contained VERSION\" +\n          \" information\");\n    }\n    if (multipleLV) {            \n      throw new IOException(\n          \"Storage directories contain multiple layout versions: \"\n              + layoutVersions);\n    }\n    // If the storage directories are with the new layout version\n    // (ie edits_<txnid>) then use the new inspector, which will ignore\n    // the old format dirs.\n    FSImageStorageInspector inspector;\n    if (NameNodeLayoutVersion.supports(\n        LayoutVersion.Feature.TXID_BASED_LAYOUT, getLayoutVersion())) {\n      inspector = new FSImageTransactionalStorageInspector(fileTypes);\n    } else {\n      inspector = new FSImagePreTransactionalStorageInspector();\n    }\n    \n    inspectStorageDirs(inspector);\n    return inspector;\n  }\n","sourceCodeStart":1106,"sourceCodeEnd":1142,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java#L1106-L1142","documentation":"NNStorage collects the layoutVersion of every readable storage directory; if they disagree, the (dir, lv) pairs are echoed in the message and startup aborts. Mixed layout versions mean the directories were last written by different Hadoop releases, and the NN cannot safely pick one storage inspector for all of them.","triggerScenarios":"A dir written by an older or newer release is added to dfs.namenode.name.dir; an interrupted rolling upgrade left some dirs at the new layout and others at the old one; a partial rollback froze mid-transition.","commonSituations":"Rolling upgrade stopped midway and the cluster is cold-restarted; copying extra name dirs between nodes running different versions; downgrade attempts abandoned halfway.","solutions":["Read the (dir, lv) pairs in the message and identify the minority-version directories.","Complete the upgrade with matching binaries: start with the newer version and run 'hdfs namenode -rollingupgrade finalize' so all dirs converge to one layoutVersion.","If the odd dirs are expendable, remove them from dfs.namenode.name.dir, restart, then clean them offline.","Never hand-edit layoutVersion - a wrong value makes the dir unparseable."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Pre-start check: all dirs must report one layoutVersion\nSet<Integer> lvs = new HashSet<>();\nfor (String loc : conf.getTrimmedStrings(\"dfs.namenode.name.dir\")) {\n  lvs.add(Integer.parseInt(parseVersion(Paths.get(stripScheme(loc), \"current\", \"VERSION\"))\n      .getOrDefault(\"layoutVersion\", \"-1\")));\n}\nif (lvs.size() > 1) {\n  throw new IllegalStateException(\"Mixed layout versions \" + lvs + \" - finish the upgrade/finalize before cold start\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Finish (finalize) rolling upgrades before any cold restart so layouts converge.","Never copy name dirs between nodes running different Hadoop versions.","Include layoutVersion in the pre-start verification output for quick audits."],"tags":["hdfs","namenode","upgrade","layout-version","storage"],"backgroundTag":"layout-version-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}