apache/hadoop · critical · InconsistentFSStateException
namespaceID is incompatible with others.
Error message
namespaceID is incompatible with others.
What it means
StorageInfo.setNamespaceID compares the namespaceID in the directory's VERSION with the namespaceID already loaded: if both are non-zero and differ, the directories belong to different namespaces and InconsistentFSStateException is thrown. An empty namespaceID is not permitted here (getProperty would already have failed on a missing key), so the mismatch is always a real conflict.
Source
Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/StorageInfo.java:231
}
/** Validate and set layout version from {@link Properties}*/
protected void setLayoutVersion(Properties props, StorageDirectory sd)
throws IncorrectVersionException, InconsistentFSStateException {
int lv = Integer.parseInt(getProperty(props, sd, "layoutVersion"));
if (lv < getServiceLayoutVersion()) { // future version
throw new IncorrectVersionException(getServiceLayoutVersion(), lv,
"storage directory " + sd.root.getAbsolutePath());
}
layoutVersion = lv;
}
/** Validate and set namespaceID version from {@link Properties}*/
protected void setNamespaceID(Properties props, StorageDirectory sd)
throws InconsistentFSStateException {
int nsId = Integer.parseInt(getProperty(props, sd, "namespaceID"));
if (namespaceID != 0 && nsId != 0 && namespaceID != nsId) {
throw new InconsistentFSStateException(sd.root,
"namespaceID is incompatible with others.");
}
namespaceID = nsId;
}
public void setServiceLayoutVersion(int lv) {
this.layoutVersion = lv;
}
public int getServiceLayoutVersion() {
return storageType == NodeType.DATA_NODE
? DataNodeLayoutVersion.getCurrentLayoutVersion()
: HdfsServerConstants.NAMENODE_LAYOUT_VERSION;
}
public Map<Integer, SortedSet<LayoutFeature>> getServiceLayoutFeatureMap() {
return storageType == NodeType.DATA_NODE? DataNodeLayoutVersion.FEATURES
: NameNodeLayoutVersion.FEATURES;View on GitHub (pinned to 2add963021)
Solutions
- Compare namespaceID across <dir>/VERSION files of all configured directories to find the mismatched one
- Remove the mismatched directory from the configuration, or wipe/reformat it so it adopts the running namespace
- After reformatting a NameNode, also clear DataNode storage directories so namespaceIDs realign on re-registration
Defensive patterns
Strategy: validation
Validate before calling
Set<Integer> ids = new HashSet<>();
for (String d : configuredDirs) {
Properties p = Storage.readPropertiesFile(new File(d, "VERSION"));
int nsId = Integer.parseInt(p.getProperty("namespaceID"));
if (nsId != 0) ids.add(nsId);
}
if (ids.size() > 1) throw new IOException("mixed namespaceIDs in storage dirs: " + ids); Prevention
- When adding a volume to an existing namespace, format/wipe it first so it adopts the running namespaceID
- After reformatting a NameNode, clear DataNode storage directories too
- Preflight-compare namespaceID across all configured dirs at daemon startup
When it happens
Trigger: Mixing a freshly formatted storage directory (new namespaceID) with directories of an existing namespace; a DataNode dir from a different namespace left in the config; NameNode was reformatted but DataNode dirs were not cleared.
Common situations: Adding a newly formatted volume to an existing set of storage dirs; reformatting the NameNode in a test loop without wiping DN storage; restoring one dir from another cluster's backup.
Related errors
- Incompatible node types: storageType={storageType} but Stora
- cluster Id is incompatible with others.
- Unknown nameservice: {}
- Configuration has multiple addresses that match local node's
- Configuration dfs.namenode.rpc-address must be suffixed with
AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22).
Data as JSON: /api/errors/18ef63437876f52a.
Report an issue: GitHub.