apache/hadoop · critical · IncorrectVersionException

storage directory " + sd.getRoot().getAbsolutePath()

Error message

storage directory " + sd.getRoot().getAbsolutePath()

What it means

During rolling-upgrade ROLLBACK startup, readProperties compares the layoutVersion in the directory's VERSION with the running software's service layout version; if storage was written by a newer release (lv > getServiceLayoutVersion()), IncorrectVersionException is thrown because old binaries cannot undo the newer layout changes. Downgrade via 'rollingupgrade rollback' is therefore not supported.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java:691

      String sbpid = props.getProperty("blockpoolID");
      setBlockPoolID(sd.getRoot(), sbpid);
    }
    setDeprecatedPropertiesForUpgrade(props);
  }

  void readProperties(StorageDirectory sd, StartupOption startupOption)
      throws IOException {
    Properties props = readPropertiesFile(sd.getVersionFile());
    if (props == null) {
      throw new IOException(
          "Properties not found  for storage directory " + sd);
    }
    if (HdfsServerConstants.RollingUpgradeStartupOption.ROLLBACK
        .matches(startupOption)) {
      int lv = Integer.parseInt(getProperty(props, sd, "layoutVersion"));
      if (lv > getServiceLayoutVersion()) {
        // we should not use a newer version for rollingUpgrade rollback
        throw new IncorrectVersionException(getServiceLayoutVersion(), lv,
            "storage directory " + sd.getRoot().getAbsolutePath());
      }
      props.setProperty("layoutVersion",
          Integer.toString(getServiceLayoutVersion()));
    }
    setFieldsFromProperties(props, sd);
  }

  /**
   * Pull any properties out of the VERSION file that are from older
   * versions of HDFS and only necessary during upgrade.
   */
  private void setDeprecatedPropertiesForUpgrade(Properties props) {
    deprecatedProperties = new HashMap<>();
    String md5 = props.getProperty(DEPRECATED_MESSAGE_DIGEST_PROPERTY);
    if (md5 != null) {
      deprecatedProperties.put(DEPRECATED_MESSAGE_DIGEST_PROPERTY, md5);
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Restart with the same or newer Hadoop version that wrote the storage, and finish the rollback from there.
2. If a real downgrade is required, use the supported downgrade path of the newer release ('rollingupgrade downgrade' with the new binaries), not old-binary rollback.
3. Verify the running binary with 'hdfs version' and compare it against the layoutVersion line in the dir's VERSION before choosing.

// Before rolling-upgrade rollback, compare on-disk layout with the running binary
int diskLv = Integer.parseInt(Files.readAllLines(versionFile).stream()
    .filter(l -> l.startsWith("layoutVersion=")).findFirst().orElseThrow()
    .split("=")[1]);
if (diskLv > HdfsServerConstants.NAMENODE_LAYOUT_VERSION) {
  throw new UnsupportedOperationException(
      "Storage layout " + diskLv + " is newer than this binary (" + HdfsServerConstants.NAMENODE_LAYOUT_VERSION
      + ") - rollback with the newer binaries or use rollingupgrade downgrade");
}

When it happens

Trigger: Running 'hdfs namenode -rollingupgrade rollback' with older-version binaries after the cluster already persisted a newer layoutVersion in VERSION - i.e. attempting a downgrade rollback after the upgrade was finalized or progressed too far.

Common situations: Aborting a failed rolling upgrade after NNs already upgraded; ops skipping the documented rollback-before-finalize order; PATH still pointing at old packages (rpm/alternatives leftovers) so the 'new' node actually runs old code.

Related errors

• Cannot rollback to a newer state. Datanode previous state: L
• Cannot rollback to storage version {prevStorage.getLayoutVer
• too many temporary directories.
• *********** Upgrade is not supported from this older versio
• Unexpected version of the file system log file: logVersion.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/fe0cafdac6711dbf. Report an issue: GitHub.