apache/hadoop · critical · InconsistentFSStateException

Incompatible node types: storageType={storageType} but Stora

Error message

Incompatible node types: storageType={storageType} but StorageDirectory type={type}

What it means

StorageInfo.checkStorageType reads the storageType property from a directory's VERSION file and compares it with the NodeType the current daemon expects (NAME_NODE vs DATA_NODE). A mismatch throws InconsistentFSStateException, because NameNode and DataNode storage layouts are incompatible. When the daemon's expected storageType is null it skips the check entirely.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageInfo.java:188

    if (props == null) {
      return;
    }
    setLayoutVersion(props, sd);
    setNamespaceID(props, sd);
    setcTime(props, sd);
    setClusterId(props, layoutVersion, sd);
    checkStorageType(props, sd);
  }
  
  /** Validate and set storage type from {@link Properties}*/
  protected void checkStorageType(Properties props, StorageDirectory sd)
      throws InconsistentFSStateException {
    if (storageType == null) { //don't care about storage type
      return;
    }
    NodeType type = NodeType.valueOf(getProperty(props, sd, "storageType"));
    if (!storageType.equals(type)) {
      throw new InconsistentFSStateException(sd.root,
          "Incompatible node types: storageType=" + storageType
          + " but StorageDirectory type=" + type);
    }
  }
  
  /** Validate and set ctime from {@link Properties}*/
  protected void setcTime(Properties props, StorageDirectory sd)
      throws InconsistentFSStateException {
    cTime = Long.parseLong(getProperty(props, sd, "cTime"));
  }

  /** Validate and set clusterId from {@link Properties}*/
  protected void setClusterId(Properties props, int layoutVersion,
      StorageDirectory sd) throws InconsistentFSStateException {
    // Set cluster ID in version that supports federation
    if (LayoutVersion.supports(getServiceLayoutFeatureMap(),
        Feature.FEDERATION, layoutVersion)) {
      String cid = getProperty(props, sd, "clusterID");

View on GitHub (pinned to 2add963021)

Solutions

1. Give each daemon role its own storage directories - never overlap dfs.namenode.name.dir and dfs.datanode.data.dir
2. Inspect <dir>/VERSION to see which node type formatted it (storageType property) and reassign the directory accordingly
3. If the directory was mis-formatted and its contents are disposable, wipe it and let the correct daemon format it on startup

Example fix

<!-- before: DataNode pointed at the NameNode dir -->
<property><name>dfs.datanode.data.dir</name><value>/data/nn</value></property>
<!-- after -->
<property><name>dfs.datanode.data.dir</name><value>/data/dn</value></property>

Properties p = Storage.readPropertiesFile(new File(dir, "VERSION"));
String diskType = p.getProperty("storageType");
if (diskType != null && !diskType.equals(expectedNodeType.name())) {
  throw new IOException(dir + " belongs to a " + diskType + "; refusing to use it as "
      + expectedNodeType);
}

When it happens

Trigger: A DataNode configured with dfs.datanode.data.dir pointing at a NameNode's storage directory (or the reverse); a directory shared between both node types; a directory formatted by one daemon type then handed to the other via config copy-paste.

Common situations: Copy-pasted hdfs-site.xml between NN and DN roles; test/docker setups reusing one volume for every role; a typo listing the NN dir in the DN data-dir setting.

Related errors

• Unknown nameservice: {}
• Configuration has multiple addresses that match local node's
• Configuration dfs.namenode.rpc-address must be suffixed with
• Cannot lock storage {root}. The directory is already locked
• cluster Id is incompatible with others.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/000d7f24443a3c46. Report an issue: GitHub.