{"record":{"id":"9327dde7f02005c3","repo":"apache/hadoop","slug":"configured-cluster-topology-should-be-org-apache-h","errorCode":null,"errorMessage":"Configured cluster topology should be org.apache.hadoop.net.NetworkTopologyWithNodeGroup","messagePattern":"Configured cluster topology should be org\\.apache\\.hadoop\\.net\\.NetworkTopologyWithNodeGroup","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java","lineNumber":50,"sourceCode":" * The replica placement strategy is adjusted to:\n * If the writer is on a datanode, the 1st replica is placed on the local \n *     node(or local node-group or on local rack), otherwise a random datanode.\n * The 2nd replica is placed on a datanode that is on a different rack with 1st\n *     replica node. \n * The 3rd replica is placed on a datanode which is on a different node-group\n *     but the same rack as the second replica node.\n */\npublic class BlockPlacementPolicyWithNodeGroup extends BlockPlacementPolicyDefault {\n\n  protected BlockPlacementPolicyWithNodeGroup() {\n  }\n\n  @Override\n  public void initialize(Configuration conf,  FSClusterStats stats,\n          NetworkTopology clusterMap, \n          Host2NodesMap host2datanodeMap) {\n    if (!(clusterMap instanceof NetworkTopologyWithNodeGroup)) {\n      throw new IllegalArgumentException(\n          \"Configured cluster topology should be \"\n              + NetworkTopologyWithNodeGroup.class.getName());\n    }\n    super.initialize(conf, stats, clusterMap, host2datanodeMap);\n  }\n\n  /**\n   * choose all good favored nodes as target.\n   * If no enough targets, then choose one replica from\n   * each bad favored node's node group.\n   * @throws NotEnoughReplicasException\n   */\n  @Override\n  protected void chooseFavouredNodes(String src, int numOfReplicas,\n      List<DatanodeDescriptor> favoredNodes,\n      Set<Node> favoriteAndExcludedNodes, long blocksize,\n      int maxNodesPerRack, List<DatanodeStorageInfo> results,\n      boolean avoidStaleNodes, EnumMap<StorageType, Integer> storageTypes)","sourceCodeStart":32,"sourceCodeEnd":68,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java#L32-L68","documentation":"Thrown by BlockPlacementPolicyWithNodeGroup.initialize when this placement policy is configured (net.topology.script.file.path / dfs.namenode.block-placement-policy or equivalent pointing at the NodeGroup-aware policy) but the cluster NetworkTopology object is not NetworkTopologyWithNodeGroup. The node-group policy requires node-group-aware topology because its replica choices depend on a third hierarchy level (node group) between rack and node.","triggerScenarios":"Config selects BlockPlacementPolicyWithNodeGroup (e.g., via dfs.blockplacement.policy or reflection in vendor distros) while dfs.namenode.network-topology / net.topology.impl defaults to plain NetworkTopology (or NetworkTopologyWithRack); mismatched topology configuration between the placement policy class and the topology implementation class.","commonSituations":"Porting cluster configs between vanilla Apache Hadoop and node-group-aware distros; setting a node-group placement policy without also setting the node-group topology implementation; leftover legacy config after an upgrade.","solutions":["Make topology consistent with the policy: configure the topology implementation as org.apache.hadoop.net.NetworkTopologyWithNodeGroup (e.g., -Dnet.topology.impl=...WithNodeGroup or the distro's equivalent topology config)","Or switch back to the default policy (BlockPlacementPolicyDefault) if node groups are not used","Search all active configs (hdfs-site.xml, core-site.xml, env-injected -D props) for block-placement and topology keys and align them","Restart NameNode after the change; verify with startup logs that the policy initialized"],"exampleFix":"<!-- before: policy without matching topology -->\n<property><name>dfs.blockplacement.policy</name>\n  <value>org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyWithNodeGroup</value></property>\n<!-- topology stays plain NetworkTopology -> IllegalArgumentException -->\n\n<!-- after: align topology implementation -->\n<property><name>net.topology.impl</name>\n  <value>org.apache.hadoop.net.NetworkTopologyWithNodeGroup</value></property>","handlingStrategy":"validation","validationCode":"// Startup config check (run before NN start)\nClass<?> topo = Class.forName(conf.get(\"net.topology.impl\", \"org.apache.hadoop.net.NetworkTopology\"));\nClass<?> policy = Class.forName(conf.get(\"dfs.blockplacement.policy\",\n    \"org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault\"));\nif (policy.getSimpleName().contains(\"WithNodeGroup\")\n    && !topo.getName().endsWith(\"NetworkTopologyWithNodeGroup\")) {\n  throw new IllegalStateException(\"Set net.topology.impl=NetworkTopologyWithNodeGroup\");\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always pair BlockPlacementPolicyWithNodeGroup with NetworkTopologyWithNodeGroup","Validate placement-policy + topology config pairs in deployment automation (Ansible/Ambari checks)","After upgrade, diff old vs new topology-related keys before first NN start","Smoke-test NN startup in staging with the exact production config set"],"tags":["hdfs","block-placement","network-topology","configuration","namenode-startup"],"backgroundTag":"topology-config-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}