apache/hadoop · error · DiskBalancerException

DATANODE_ID_MISMATCH

DATANODE_ID_MISMATCH

Error message

Plan was generated for another node.

What it means

Every diskbalancer plan embeds the DataNode UUID it was computed for. verifyNodeUUID() rejects the submission with Result.DATANODE_ID_MISMATCH when plan.getNodeUUID() is null or does not equal the local DataNode's UUID. Volume-usage data in the plan is only meaningful on the node where it was measured.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancer.java:502

      }
      String errorString = "Plan was generated more than " + planValidity
          + " ago";
      LOG.error("Disk Balancer - " + errorString);
      throw new DiskBalancerException(errorString,
          DiskBalancerException.Result.OLD_PLAN_SUBMITTED);
    }
  }

  /**
   * Verify Node UUID.
   *
   * @param plan - Node Plan
   */
  private void verifyNodeUUID(NodePlan plan) throws DiskBalancerException {
    if ((plan.getNodeUUID() == null) ||
        !plan.getNodeUUID().equals(this.dataNodeUUID)) {
      LOG.error("Disk Balancer - Plan was generated for another node.");
      throw new DiskBalancerException(
          "Plan was generated for another node.",
          DiskBalancerException.Result.DATANODE_ID_MISMATCH);
    }
  }

  /**
   * Convert a node plan to DiskBalancerWorkItem that Datanode can execute.
   *
   * @param plan - Node Plan
   */
  private void createWorkPlan(NodePlan plan) throws DiskBalancerException {
    Preconditions.checkState(lock.isHeldByCurrentThread());

    // Cleanup any residual work in the map.
    workMap.clear();
    Map<String, String> storageIDToVolBasePathMap =
        getStorageIDToVolumeBasePathMap();

View on GitHub (pinned to 2add963021)

Solutions

1. Regenerate the plan for this exact node: `hdfs diskbalancer -plan <this-host>:<ipc-port>` and submit that file.
2. Compare the plan's nodeID/nodeUUID with the DataNode's UUID (DataNode JMX or the VERSION file's datanodeUuid) to confirm the mismatch.
3. If the DataNode was reformatted, discard all old plans — they are permanently invalid for the new UUID.
4. Fix automation to generate a per-node plan instead of sharing one plan across nodes.

Example fix

// before: reuse a plan generated for another DataNode
diskBalancer.submitPlan(planID, version, planString, false);

// after: verify the plan targets this node first
NodePlan p = NodePlan.parseJson(planString);
if (p.getNodeUUID() == null || !p.getNodeUUID().equals(datanode.getDatanodeUuid())) {
  planString = generateFreshPlan(datanode); // plan for THIS node
}
diskBalancer.submitPlan(planID, version, planString, false);

NodePlan p = NodePlan.parseJson(planString);
String localUuid = dataNode.getDatanodeUuid();
if (p.getNodeUUID() == null || !p.getNodeUUID().equals(localUuid)) {
  // plan targets a different DataNode: regenerate for this node
}

try {
  diskBalancer.submitPlan(planID, version, plan, false);
} catch (DiskBalancerException e) {
  if (e.getResult() == DiskBalancerException.Result.DATANODE_ID_MISMATCH) {
    // plan belongs to another node: regenerate locally, do not retry the same file
  }
}

When it happens

Trigger: Submitting a plan generated by `hdfs diskbalancer -plan other-host:port` to this DataNode; a plan whose nodeUUID field was removed or blanked by hand-editing; a DataNode whose storage was reformatted so its Datanode UUID changed after the plan was generated.

Common situations: Automation that generates one plan and submits it to every DataNode; cluster re-format or data-dir wipe changing the datanode UUID; multi-homed hosts where hostname resolution differs between plan generation and submission.

Related errors

• MALFORMED_PLAN
• INVALID_MOVE
• Not a valid Boolean value for {property}
• Root {}: DatanodeUuid={}, does not match {} from other Stora
• INVALID_PLAN

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/778078e510cd5f68. Report an issue: GitHub.