apache/hadoop · error · IOException

Incompatible version for state: expecting state version

Error message

Incompatible version for state: expecting state version {}, but loading version {}

What it means

The LevelDB store records a schema version row (DB_SCHEMA_VERSION_KEY) and compares it with the current code version at startup via checkVersion. Equal versions pass; compatible versions are silently upgraded by writing the new version; an incompatible version throws this IOException so the server refuses to run against state it cannot interpret. The loaded version is also logged as 'Loaded state version info' for comparison.

Solutions

1. Run the Hadoop version compatible with the stored state (compare the logged loaded version with the expected one in the message).
2. If the stored token state is disposable, stop JHS and remove the LevelDB directory; it is recreated at the current version and clients re-authenticate.
3. Back up the directory before deleting it.
4. Plan upgrades through compatible intermediate versions instead of downgrading.

try {
  // JHS startup with existing leveldb state
} catch (IOException e) {
  if (e.getMessage().startsWith("Incompatible version for state")) {
    // run the matching Hadoop version, or stop JHS and remove the state dir to reinitialize
  }
}

When it happens

Trigger: Downgrading JHS to an older Hadoop whose state version is incompatible with the stored one; jumping across incompatible major versions; pointing mapreduce.jobhistory.recovery.store.leveldb.path at a database created by different code.

Common situations: Rollback after a failed upgrade; repurposing a state directory; version skew between nodes in mixed-version clusters.

Related errors

• Error loading token master key from
• Error loading token state from
• Could not rename to
• Could not rename to
• already exists

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/63caed5cc375d877. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryServerLeveldbStateStoreService.java:363

   * 2) Any incompatible change of state-store is a major upgrade, and any
   *    compatible change of state-store is a minor upgrade.
   * 3) Within a minor upgrade, say 1.1 to 1.2:
   *    overwrite the version info and proceed as normal.
   * 4) Within a major upgrade, say 1.2 to 2.0:
   *    throw exception and indicate user to use a separate upgrade tool to
   *    upgrade state or remove incompatible old state.
   */
  private void checkVersion() throws IOException {
    Version loadedVersion = loadVersion();
    LOG.info("Loaded state version info " + loadedVersion);
    if (loadedVersion.equals(getCurrentVersion())) {
      return;
    }
    if (loadedVersion.isCompatibleTo(getCurrentVersion())) {
      LOG.info("Storing state version info " + getCurrentVersion());
      storeVersion();
    } else {
      throw new IOException(
        "Incompatible version for state: expecting state version "
            + getCurrentVersion() + ", but loading version " + loadedVersion);
    }
  }
}

View on GitHub (pinned to 2add963021)