apache/hadoop · critical · InconsistentFSStateException

Incompatible node types: storageType=

Error message

Incompatible node types: storageType={storageType} but StorageDirectory type={type}

What it means

StorageInfo.checkStorageType reads the storageType property from a directory's VERSION file and compares it with the NodeType the current daemon expects (NAME_NODE vs DATA_NODE). A mismatch throws InconsistentFSStateException, because NameNode and DataNode storage layouts are incompatible. When the daemon's expected storageType is null it skips the check entirely.

Solutions

Give each daemon role its own storage directories - never overlap dfs.namenode.name.dir and dfs.datanode.data.dir
Inspect <dir>/VERSION to see which node type formatted it (storageType property) and reassign the directory accordingly
If the directory was mis-formatted and its contents are disposable, wipe it and let the correct daemon format it on startup

Example fix

<!-- before: DataNode pointed at the NameNode dir -->
<property><name>dfs.datanode.data.dir</name><value>/data/nn</value></property>
<!-- after -->
<property><name>dfs.datanode.data.dir</name><value>/data/dn</value></property>

Defensive patterns

Strategy: validation

Validate before calling

Properties p = Storage.readPropertiesFile(new File(dir, "VERSION"));
String diskType = p.getProperty("storageType");
if (diskType != null && !diskType.equals(expectedNodeType.name())) {
  throw new IOException(dir + " belongs to a " + diskType + "; refusing to use it as "
      + expectedNodeType);
}

Prevention

Lint storage-dir configurations: NN and DN dir lists must be disjoint
Check the storageType property in a directory's VERSION before assigning it to a daemon role
Use role-specific config templates for NameNode and DataNode hosts

When it happens

Trigger: A DataNode configured with dfs.datanode.data.dir pointing at a NameNode's storage directory (or the reverse); a directory shared between both node types; a directory formatted by one daemon type then handed to the other via config copy-paste.

Common situations: Copy-pasted hdfs-site.xml between NN and DN roles; test/docker setups reusing one volume for every role; a typo listing the NN dir in the DN data-dir setting.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/000d7f24443a3c46. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageInfo.java:188

    if (props == null) {
      return;
    }
    setLayoutVersion(props, sd);
    setNamespaceID(props, sd);
    setcTime(props, sd);
    setClusterId(props, layoutVersion, sd);
    checkStorageType(props, sd);
  }
  
  /** Validate and set storage type from {@link Properties}*/
  protected void checkStorageType(Properties props, StorageDirectory sd)
      throws InconsistentFSStateException {
    if (storageType == null) { //don't care about storage type
      return;
    }
    NodeType type = NodeType.valueOf(getProperty(props, sd, "storageType"));
    if (!storageType.equals(type)) {
      throw new InconsistentFSStateException(sd.root,
          "Incompatible node types: storageType=" + storageType
          + " but StorageDirectory type=" + type);
    }
  }
  
  /** Validate and set ctime from {@link Properties}*/
  protected void setcTime(Properties props, StorageDirectory sd)
      throws InconsistentFSStateException {
    cTime = Long.parseLong(getProperty(props, sd, "cTime"));
  }

  /** Validate and set clusterId from {@link Properties}*/
  protected void setClusterId(Properties props, int layoutVersion,
      StorageDirectory sd) throws InconsistentFSStateException {
    // Set cluster ID in version that supports federation
    if (LayoutVersion.supports(getServiceLayoutFeatureMap(),
        Feature.FEDERATION, layoutVersion)) {
      String cid = getProperty(props, sd, "clusterID");

View on GitHub (pinned to 2add963021)