What causes "Invalid or mis-matched hash." in apache/hadoop?

Regenerate the plan with 'hdfs diskbalancer -plan ' and execute the untouched file - the CLI passes the matching hash automatically

apache/hadoop · error · DiskBalancerException

INVALID_PLAN_HASH

INVALID_PLAN_HASH

Error message

Invalid or mis-matched hash.

What it means

DiskBalancerException with Result.INVALID_PLAN_HASH from DiskBalancer.verifyPlanHash: the planID (expected to be the 40-hex-char SHA-1 of the plan bytes) is null, not exactly 40 characters, or does not match DigestUtils.sha1Hex(plan) computed on the DataNode. This integrity check guarantees the executed plan is byte-for-byte the one the planner signed, preventing corrupted or tampered plans from moving data.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancer.java:457

   * @param planID - SHA-1 Hex Bytes
   * @param plan   - Plan String
   * @throws DiskBalancerException
   */
  private NodePlan verifyPlanHash(String planID, String plan)
      throws DiskBalancerException {
    final long sha1Length = 40;
    if (plan == null || plan.length() == 0) {
      LOG.error("Disk Balancer -  Invalid plan.");
      throw new DiskBalancerException("Invalid plan.",
          DiskBalancerException.Result.INVALID_PLAN);
    }

    if ((planID == null) ||
        (planID.length() != sha1Length) ||
        !DigestUtils.sha1Hex(plan.getBytes(StandardCharsets.UTF_8))
            .equalsIgnoreCase(planID)) {
      LOG.error("Disk Balancer - Invalid plan hash.");
      throw new DiskBalancerException("Invalid or mis-matched hash.",
          DiskBalancerException.Result.INVALID_PLAN_HASH);
    }

    try {
      return NodePlan.parseJson(plan);
    } catch (IOException ex) {
      throw new DiskBalancerException("Parsing plan failed.", ex,
          DiskBalancerException.Result.MALFORMED_PLAN);
    }
  }

  /**
   * Verifies that this plan is not older than 24 hours.
   *
   * @param plan - Node Plan
   */
  private void verifyTimeStamp(NodePlan plan) throws DiskBalancerException {
    long now = Time.now();

View on GitHub (pinned to 2add963021)

Solutions

Regenerate the plan with 'hdfs diskbalancer -plan <node>' and execute the untouched file - the CLI passes the matching hash automatically
If the plan was edited intentionally, recompute and use the new SHA-1: sha1sum <planfile> (hex, lowercase) and pass that as the planID
Verify the file arrived intact: compare sha1sum on the machine that generated and the one executing the plan
For API callers, ensure planID = DigestUtils.sha1Hex(planData.getBytes(StandardCharsets.UTF_8)) of the exact string submitted

Example fix

# before: plan JSON edited/reformatted -> INVALID_PLAN_HASH
vim /system/diskbalancer/node-plan.json   # pretty-printed, hash now stale
hdfs diskbalancer -execute /system/diskbalancer/node-plan.json

# after: regenerate (or recompute hash) so SHA-1 matches the bytes
hdfs diskbalancer -plan <datanode>          # fresh file + matching hash
# or: sha1sum /system/diskbalancer/node-plan.json  -> use that digest as planID

Defensive patterns

Strategy: validation

Validate before calling

// Compute and verify the SHA-1 exactly like the DataNode does, before submit
String planData = new String(Files.readAllBytes(Paths.get(planFile)),
    StandardCharsets.UTF_8);
String expected = DigestUtils.sha1Hex(planData.getBytes(StandardCharsets.UTF_8));
if (planId == null || planId.length() != 40 || !expected.equalsIgnoreCase(planId)) {
  throw new IllegalArgumentException(
      "planID must be the 40-hex SHA-1 of the plan bytes; computed: " + expected);
}
diskBalancer.submitPlan(planId, planVersion, planFile, planData, force);

Try / catch

try {
  diskBalancer.submitPlan(planId, planVersion, planFile, planData, force);
} catch (DiskBalancerException e) {
  if (e.getResult() == DiskBalancerException.Result.INVALID_PLAN_HASH) {
    // plan bytes and ID diverge: either regenerate plan+ID together, or recompute
    // sha1Hex(planData) and resubmit with that as planID - never hand-edit plans
  }
}

Prevention

Never edit/reformat a generated plan file; regenerate it instead so hash and bytes stay paired
Transfer plan files byte-exact (binary mode, checksum after copy: sha1sum on both ends)
When calling submitPlan programmatically, always derive planID via DigestUtils.sha1Hex(planData.getBytes(UTF_8)) from the exact payload you submit

When it happens

Trigger: 'hdfs diskbalancer -execute <planfile>' where the planID argument/plan-file hash differs from SHA-1 of the plan content: file corrupted or edited after generation (whitespace/line-ending changes), wrong hash copied from another plan, truncated file transfer, or planID typoed/truncated on the command line or API call.

Common situations: Editing a generated plan JSON (even reformatting) then executing it; copying plan files through tools that alter encoding/line endings; passing the planID of a different node's plan; programmatic submitPlan computing the hash over a different string than the plan payload sent.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e99f635636ea272d. Report an issue: GitHub.