apache/hadoop · error · InconsistentFSStateException

previous and previous.tmp cannot exist together.

Error message

previous and previous.tmp cannot exist together.

What it means

When previous.tmp exists, an upgrade completion is pending (previous.tmp is the renamed-away old current/, destined to become previous/; normal outcomes are COMPLETE_UPGRADE with current/ present or RECOVER_UPGRADE without). If previous/ already exists too, the rename target is occupied — a duplicate 'previous' — and Storage.analyzeStorage throws InconsistentFSStateException('previous and previous.tmp cannot exist together.').

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java:762

                                               "too many temporary directories.");

      // # of temp dirs == 1 should either recover or complete a transition
      if (hasCheckpointTmp) {
        return hasCurrent ? StorageState.COMPLETE_CHECKPOINT
                          : StorageState.RECOVER_CHECKPOINT;
      }

      if (hasFinalizedTmp) {
        if (hasPrevious)
          throw new InconsistentFSStateException(root,
                                                 STORAGE_DIR_PREVIOUS + " and " + STORAGE_TMP_FINALIZED
                                                 + "cannot exist together.");
        return StorageState.COMPLETE_FINALIZE;
      }

      if (hasPreviousTmp) {
        if (hasPrevious)
          throw new InconsistentFSStateException(root,
                                                 STORAGE_DIR_PREVIOUS + " and " + STORAGE_TMP_PREVIOUS
                                                 + " cannot exist together.");
        if (hasCurrent)
          return StorageState.COMPLETE_UPGRADE;
        return StorageState.RECOVER_UPGRADE;
      }
      
      assert hasRemovedTmp : "hasRemovedTmp must be true";
      if (!(hasCurrent ^ hasPrevious))
        throw new InconsistentFSStateException(root,
                                               "one and only one directory " + STORAGE_DIR_CURRENT 
                                               + " or " + STORAGE_DIR_PREVIOUS 
                                               + " must be present when " + STORAGE_TMP_REMOVED
                                               + " exists.");
      if (hasCurrent)
        return StorageState.COMPLETE_ROLLBACK;
      return StorageState.RECOVER_ROLLBACK;
    }

View on GitHub (pinned to 2add963021)

Solutions

  1. Stop the daemon and back up the storage directory
  2. Keep exactly one 'previous' artifact: to let the upgrade complete, remove previous/ and keep previous.tmp (boot runs COMPLETE_UPGRADE when current/ exists); to keep the existing previous/, remove previous.tmp
  3. Restart the daemon and verify it reaches NORMAL

Example fix

# previous/ AND previous.tmp both present after crashed upgrade
# goal: complete upgrade (current/ present)
mv /dfs/nn/previous /backup/previous.old
# restart namenode -> COMPLETE_UPGRADE renames previous.tmp -> previous
Defensive patterns

Strategy: try-catch

Validate before calling

File root = sd.getRoot();
boolean hasPrev = new File(root, "previous").exists();
boolean hasPrevTmp = new File(root, "previous.tmp").exists();
if (hasPrev && hasPrevTmp) {
  throw new IOException("previous/ and previous.tmp coexist in " + root
      + "; remove the stale one (keep previous.tmp to complete the upgrade)");
}

Try / catch

try {
  daemon.start();
} catch (InconsistentFSStateException e) {
  if (e.getMessage().contains("previous.tmp")) {
    haltWithRunbook("Crashed upgrade completion: remove previous/ (complete) or "
        + "previous.tmp (keep existing previous/); back up first");
  } else throw e;
}

Prevention

When it happens

Trigger: A crash during the upgrade completion window together with a previous/ that appeared from outside the state machine: restored from backup, copied in by hand, or left by an earlier interrupted upgrade cycle.

Common situations: Repeatedly interrupted upgrades; operators copying previous/ back 'to be safe' before restarting; storage snapshot reverted to a mid-upgrade point while another marker survived.

Related errors


AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b02150175f323add. Report an issue: GitHub.