apache/hadoop · error · IllegalArgumentException

Unexpected data node {} at an illegal network location

Error message

Unexpected data node {} at an illegal network location

What it means

During add(), getNodeForNetworkLocation(node) (NetworkTopology.java:193) returns getNode(node.getNetworkLocation()) — the node already registered at the position where the new leaf's rack must sit. That position must be an InnerNode; if a data (leaf) node already occupies it, the topology is self-contradictory and the add is rejected.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetworkTopology.java:151

  public void add(Node node) {
    if (node==null) return;
    int newDepth = NodeBase.locationToDepth(node.getNetworkLocation()) + 1;
    netlock.writeLock().lock();
    try {
      if( node instanceof InnerNode ) {
        throw new IllegalArgumentException(
          "Not allow to add an inner node: "+NodeBase.getPath(node));
      }
      if ((depthOfAllLeaves != -1) && (depthOfAllLeaves != newDepth)) {
        LOG.error("Error: can't add leaf node {} at depth {} to topology:{}\n",
            NodeBase.getPath(node), newDepth, this);
        throw new InvalidTopologyException("Failed to add " + NodeBase.getPath(node) +
            ": You cannot have a rack and a non-rack node at the same " +
            "level of the network topology.");
      }
      Node rack = getNodeForNetworkLocation(node);
      if (rack != null && !(rack instanceof InnerNode)) {
        throw new IllegalArgumentException("Unexpected data node " 
                                           + node.toString() 
                                           + " at an illegal network location");
      }
      if (clusterMap.add(node)) {
        LOG.info("Adding a new node: "+NodeBase.getPath(node));
        if (rack == null) {
          incrementRacks();
        }
        interAddNodeWithEmptyRack(node);
        if (depthOfAllLeaves == -1) {
          depthOfAllLeaves = node.getLevel();
        }
      }
      LOG.debug("NetworkTopology became:\n{}", this);
    } finally {
      netlock.writeLock().unlock();
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Run `hdfs dfsadmin -printTopology` and find the data node sitting where a rack should be; correct that host's mapping to a real rack path
2. Ensure no node's network location equals another node's name or full path — hosts and racks must occupy different levels
3. Fix the mapping data/script and re-register the affected nodes

Example fix

// before: mapping collides — host 'dn9' is itself a datanode
dn1 -> /dn9

// after: racks and host names live on different levels
dn1 -> /rack1
dn9 -> /rack1

Node rack = topology.getNode(node.getNetworkLocation());
if (rack != null && !(rack instanceof InnerNode)) {
  throw new IllegalStateException("Leaf " + rack.getName()
      + " occupies rack position " + node.getNetworkLocation());
}
topology.add(node);

static boolean rackPositionIsFreeOrInner(NetworkTopology t, Node candidate) {
  Node at = t.getNode(candidate.getNetworkLocation());
  return at == null || at instanceof InnerNode;
}

catch (IllegalArgumentException e) {
  if (e.getMessage().contains("illegal network location")) {
    // mapping collision: re-check topology and re-register with corrected location
  }
}

When it happens

Trigger: A leaf registered with a shallow location makes its full path look like a rack path (e.g. a node named 'r1' registered at root has path /r1); when another node is then mapped to location /r1, getNode('/r1') returns that leaf, not an InnerNode, and the exception fires.

Common situations: StaticMapping (net.topology.node.switch.mapping.impl) key-value data where one host's location value equals another host's name; topology scripts returning a bare hostname as a location for some nodes; name/location collisions after re-registering nodes with edited mappings.

Related errors

• Failed to add {}: You cannot have a rack and a non-rack node
• Unexpected data node {} at an illegal network location
• Network Location is null
• Network Location path does not start with /: {}
• Errors on getting mount table loader class. The fs.viewfs.mo

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/f30fa74a07550132. Report an issue: GitHub.