apache/hadoop · error · DiskBalancerException

MALFORMED_PLAN

MALFORMED_PLAN

Error message

Parsing plan failed.

What it means

Thrown by the DataNode-side DiskBalancer when NodePlan.parseJson() raises an IOException while parsing a submitted plan. The check runs after the planID sha1 verification, so the hash matched the bytes but the bytes themselves are not a valid diskbalancer NodePlan JSON document. The DiskBalancerException carries Result.MALFORMED_PLAN and chains the original IOException as its cause.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancer.java:464

    if (plan == null || plan.length() == 0) {
      LOG.error("Disk Balancer -  Invalid plan.");
      throw new DiskBalancerException("Invalid plan.",
          DiskBalancerException.Result.INVALID_PLAN);
    }

    if ((planID == null) ||
        (planID.length() != sha1Length) ||
        !DigestUtils.sha1Hex(plan.getBytes(StandardCharsets.UTF_8))
            .equalsIgnoreCase(planID)) {
      LOG.error("Disk Balancer - Invalid plan hash.");
      throw new DiskBalancerException("Invalid or mis-matched hash.",
          DiskBalancerException.Result.INVALID_PLAN_HASH);
    }

    try {
      return NodePlan.parseJson(plan);
    } catch (IOException ex) {
      throw new DiskBalancerException("Parsing plan failed.", ex,
          DiskBalancerException.Result.MALFORMED_PLAN);
    }
  }

  /**
   * Verifies that this plan is not older than 24 hours.
   *
   * @param plan - Node Plan
   */
  private void verifyTimeStamp(NodePlan plan) throws DiskBalancerException {
    long now = Time.now();
    long planTime = plan.getTimeStamp();

    if ((planTime + planValidityInterval) < now) {
      String planValidity = config.get(
          DFSConfigKeys.DFS_DISK_BALANCER_PLAN_VALID_INTERVAL,
          DFSConfigKeys.DFS_DISK_BALANCER_PLAN_VALID_INTERVAL_DEFAULT);
      if (planValidity.matches("[0-9]$")) {

View on GitHub (pinned to 2add963021)

Solutions

1. Regenerate the plan on the target node with `hdfs diskbalancer -plan <datanode-host:ipc-port>` and submit the fresh file unchanged.
2. Validate the file parses as JSON before submitting: `jq . <planfile>.plan.json` must succeed.
3. Confirm you pass the file's full contents (not its path) and that the planID equals sha1Hex of exactly those bytes.
4. Align the `hdfs` client version used for -plan with the DataNode version that will execute the plan.

Example fix

// before: submit whatever string was read earlier
diskBalancer.submitPlan(planID, version, planString, false);

// after: parse and hash-check before submitting
NodePlan parsed;
try {
  parsed = NodePlan.parseJson(planString);
} catch (IOException e) {
  throw new IllegalArgumentException("plan is not valid NodePlan JSON; regenerate it", e);
}
if (!DigestUtils.sha1Hex(planString.getBytes(StandardCharsets.UTF_8))
    .equalsIgnoreCase(planID)) {
  throw new IllegalArgumentException("planID does not match plan content hash");
}
diskBalancer.submitPlan(planID, version, planString, false);

// Run before submitPlan
try {
  NodePlan.parseJson(planString);
} catch (IOException e) {
  // plan content is malformed: regenerate instead of submitting
}
if (!DigestUtils.sha1Hex(planString.getBytes(StandardCharsets.UTF_8)).equalsIgnoreCase(planID)) {
  // planID and plan content disagree: re-derive planID or regenerate plan
}

try {
  diskBalancer.submitPlan(planID, version, plan, false);
} catch (DiskBalancerException e) {
  if (e.getResult() == DiskBalancerException.Result.MALFORMED_PLAN) {
    // content corrupt: regenerate the plan, never resubmit the same bytes
  } else {
    throw e;
  }
}

When it happens

Trigger: Calling DiskBalancer.submitPlan(planID, version, plan, skipDateCheck) with a plan string that is truncated, hand-edited, empty, non-JSON, or generated by an incompatible HDFS version whose NodePlan schema differs. Also produced by passing the plan file's path instead of its contents, or by a transfer step that corrupts the file (shell heredoc mangling, encoding change).

Common situations: Manually editing plan.json to tweak bytesToMove and breaking JSON syntax; copying plan files between hosts with truncation; running `hdfs diskbalancer -plan` with a client whose version differs from the DataNode executing the plan; scripts that read the wrong file or only part of it.

Related errors

• Not a valid Boolean value for {property}
• OLD_PLAN_SUBMITTED
• DATANODE_ID_MISMATCH
• INVALID_VOLUME
• INVALID_MOVE

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0cec87865b92b2c4. Report an issue: GitHub.