apache/hadoop · critical · InconsistentFSStateException

version file in current directory is missing.

Error message

version file in current directory is missing.

What it means

In Storage.analyzeStorage, when no transition temp dirs exist and current/VERSION is missing (hasCurrent == false, Storage.java:716-717) but previous/ does exist, the directory is in an impossible state and InconsistentFSStateException('version file in current directory is missing.') aborts startup. previous/ is only supposed to coexist with a complete current/ (the window between upgrade completion and finalize); without current/, the daemon refuses to guess.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java:732

      // check whether current directory is valid
      File versionFile = getVersionFile();
      boolean hasCurrent = versionFile.exists();

      // check which directories exist
      boolean hasPrevious = getPreviousDir().exists();
      boolean hasPreviousTmp = getPreviousTmp().exists();
      boolean hasRemovedTmp = getRemovedTmp().exists();
      boolean hasFinalizedTmp = getFinalizedTmp().exists();
      boolean hasCheckpointTmp = getLastCheckpointTmp().exists();

      if (!(hasPreviousTmp || hasRemovedTmp
          || hasFinalizedTmp || hasCheckpointTmp)) {
        // no temp dirs - no recovery
        if (hasCurrent)
          return StorageState.NORMAL;
        if (hasPrevious)
          throw new InconsistentFSStateException(root,
                              "version file in current directory is missing.");
        if (checkCurrentIsEmpty) {
          checkEmptyCurrent();
        }
        return StorageState.NOT_FORMATTED;
      }

      if ((hasPreviousTmp?1:0) + (hasRemovedTmp?1:0)
          + (hasFinalizedTmp?1:0) + (hasCheckpointTmp?1:0) > 1)
        // more than one temp dirs
        throw new InconsistentFSStateException(root,
                                               "too many temporary directories.");

      // # of temp dirs == 1 should either recover or complete a transition
      if (hasCheckpointTmp) {
        return hasCurrent ? StorageState.COMPLETE_CHECKPOINT
                          : StorageState.RECOVER_CHECKPOINT;
      }

View on GitHub (pinned to 2add963021)

Solutions

  1. Inspect previous/ — it typically holds the last complete pre-upgrade layout: recover by completing the interrupted transition (e.g., restore current/ from a checkpoint/fsimage backup) rather than deleting anything
  2. If the namespace on this directory is disposable, move previous/ aside as a backup and re-format the directory
  3. For a NameNode, prefer restoring from the latest fsimage/checkpoint backup over hand-editing storage state

Example fix

# diagnosis
ls /dfs/nn/current/VERSION /dfs/nn/previous   # VERSION gone, previous/ present

# recovery options (stop daemons first, back up the whole dir)
mv /dfs/nn/previous /backup/previous
sudo -u hdfs hdfs namenode -format            # only if data is disposable
Defensive patterns

Strategy: try-catch

Validate before calling

// startup preflight for each storage dir
File current = new File(sd.getRoot(), Storage.STORAGE_DIR_CURRENT);
File previous = new File(sd.getRoot(), Storage.STORAGE_DIR_PREVIOUS);
boolean hasCurrent = new File(current, "VERSION").exists(); // mirrors Storage.java:716-717
if (!hasCurrent && previous.exists()) {
  throw new IOException("Inconsistent storage: previous/ exists without current/VERSION; "
      + "recover the interrupted transition or move previous/ aside and format");
}

Try / catch

try {
  daemon.start();
} catch (InconsistentFSStateException e) {
  if (e.getMessage().contains("version file in current directory is missing")) {
    haltWithRunbook("Storage dir " + e.getDir() + ": recover previous/ (crashed upgrade/rollback) "
        + "or back up and reformat; do not delete previous/ blindly");
  } else throw e;
}

Prevention

When it happens

Trigger: NameNode/DataNode/JournalNode startup after an interrupted upgrade/rollback destroyed current/ while previous/ survived, or after VERSION in current/ was deleted/pruned by hand.

Common situations: Crash during upgrade finalization or rollback; operators deleting current/ to 'reset' a node while leaving previous/; disk corruption eating current/ but not previous/; partial rsync of a storage dir.

Related errors


AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4a3b38d32085cee3. Report an issue: GitHub.