apache/hadoop · critical · InconsistentFSStateException

file VERSION has " + name + " missing.

Error message

file VERSION has " + name + " missing.

What it means

StorageInfo.getProperty looks up a required field (layoutVersion, storageType, namespaceID, clusterID, cTime) in the VERSION file's Properties; a null result means the key is absent and InconsistentFSStateException is thrown naming the file and the missing key. A VERSION file missing keys is truncated, hand-edited, or corrupted.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageInfo.java:256

    this.layoutVersion = lv;
  }

  public int getServiceLayoutVersion() {
    return storageType == NodeType.DATA_NODE
        ? DataNodeLayoutVersion.getCurrentLayoutVersion()
        : HdfsServerConstants.NAMENODE_LAYOUT_VERSION;
  }

  public Map<Integer, SortedSet<LayoutFeature>> getServiceLayoutFeatureMap() {
    return storageType == NodeType.DATA_NODE? DataNodeLayoutVersion.FEATURES
        : NameNodeLayoutVersion.FEATURES;
  }
  
  protected static String getProperty(Properties props, StorageDirectory sd,
      String name) throws InconsistentFSStateException {
    String property = props.getProperty(name);
    if (property == null) {
      throw new InconsistentFSStateException(sd.root, "file "
          + STORAGE_FILE_VERSION + " has " + name + " missing.");
    }
    return property;
  }

  public static Properties readPropertiesFile(File from) throws IOException {
    if (from == null) {
      return null;
    }
    RandomAccessFile file = new RandomAccessFile(from, "rws");
    FileInputStream in = null;
    Properties props = new Properties();
    try {
      in = new FileInputStream(file.getFD());
      file.seek(0);
      props.load(in);
    } finally {
      if (in != null) {

View on GitHub (pinned to 2add963021)

Solutions

1. cat <dir>/VERSION and compare its keys against a healthy directory of the same namespace
2. Restore the complete VERSION from backup or replicate it from another storage dir of the same namespace (all fields must match exactly)
3. If no trustworthy copy exists, back up and reformat the directory

Properties p = Storage.readPropertiesFile(new File(dir, "VERSION"));
for (String key : new String[]{"layoutVersion", "storageType", "namespaceID", "cTime"}) {
  if (p.getProperty(key) == null)
    throw new IOException("VERSION in " + dir + " is missing " + key + "; corrupted?");
}

try {
  storage.readProperties(sd);
} catch (InconsistentFSStateException e) {
  if (e.getMessage() != null && e.getMessage().contains("missing")) {
    // quarantine the directory and restore VERSION from a healthy sibling before restart
  }
  throw e;
}

When it happens

Trigger: A VERSION file truncated by a crash or full disk during format/write; keys removed by manual editing; a VERSION file written by a much older release lacking the key (e.g. pre-federation file with no clusterID); bit rot or partial restore from backup.

Common situations: Formatting interrupted by power loss; operators hand-editing VERSION to fix other errors; storage restored from an inconsistent backup; flash/NFS corruption.

Related errors

• file VERSION is invalid.
• file VERSION is invalid.
• Properties not found for storage directory " + sd
• No storage directories contained VERSION information
• Same delegation token being added twice; invalid entry in fs

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b78faf9d7425305f. Report an issue: GitHub.