apache/hadoop · error · IOException

Incompatible version for state DB schema: expecting DB schem

Error message

Incompatible version for state DB schema: expecting DB schema version {currentVersion}, but loading version {loadedVersion}

What it means

Error "Incompatible version for state DB schema: expecting DB schema version {currentVersion}, but loading version {loadedVersion}" thrown in apache/hadoop.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java:587

   * 2) Any incompatible change of DB schema is a major upgrade, and any
   *    compatible change of DB schema is a minor upgrade.
   * 3) Within a minor upgrade, say 1.1 to 1.2:
   *    overwrite the version info and proceed as normal.
   * 4) Within a major upgrade, say 1.2 to 2.0:
   *    throw exception and indicate user to use a separate upgrade tool to
   *    upgrade shuffle info or remove incompatible old state.
   */
  private void checkVersion() throws IOException {
    Version loadedVersion = loadVersion();
    LOG.info("Loaded state DB schema version info " + loadedVersion);
    if (loadedVersion.equals(getCurrentVersion())) {
      return;
    }
    if (loadedVersion.isCompatibleTo(getCurrentVersion())) {
      LOG.info("Storing state DB schema version info " + getCurrentVersion());
      storeVersion();
    } else {
      throw new IOException(
        "Incompatible version for state DB schema: expecting DB schema version " 
            + getCurrentVersion() + ", but loading version " + loadedVersion);
    }
  }

  private void addJobToken(JobID jobId, String user,
      Token<JobTokenIdentifier> jobToken) {
    userRsrc.put(jobId.toString(), user);
    secretManager.addTokenForJob(jobId.toString(), jobToken);
    LOG.info("Added token for " + jobId.toString());
  }

  private void recoverJobShuffleInfo(String jobIdStr, byte[] data)
      throws IOException {
    JobID jobId;
    try {
      jobId = JobID.forName(jobIdStr);
    } catch (IllegalArgumentException e) {

View on GitHub (pinned to 2add963021)

Solutions

1. The stored state DB schema version does not match this Hadoop version: delete the old recovery DB (losing recovery state) or run the matching Hadoop version.
2. Plan rolling upgrades so NodeManagers run a version compatible with the persisted schema.

When it happens

Trigger: Thrown by ShuffleHandler when the stored state DB schema version differs from the version the running shuffle handler expects, typically after a Hadoop upgrade. Delete the old recovery state store so it is recreated with the current schema version.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/51dcc0025d284932. Report an issue: GitHub.