apache/hadoop · error · DiskBalancerException

OLD_PLAN_SUBMITTED

OLD_PLAN_SUBMITTED

Error message

Plan was generated more than {} ago

What it means

DiskBalancer refuses to execute a plan older than dfs.disk.balancer.plan.valid.interval (default '2d'). A plan is a point-in-time snapshot of volume usage; running a stale one would move data according to outdated numbers. verifyTimeStamp() throws DiskBalancerException with Result.OLD_PLAN_SUBMITTED when plan timestamp + validity interval is still in the past.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DiskBalancer.java:488

   * Verifies that this plan is not older than 24 hours.
   *
   * @param plan - Node Plan
   */
  private void verifyTimeStamp(NodePlan plan) throws DiskBalancerException {
    long now = Time.now();
    long planTime = plan.getTimeStamp();

    if ((planTime + planValidityInterval) < now) {
      String planValidity = config.get(
          DFSConfigKeys.DFS_DISK_BALANCER_PLAN_VALID_INTERVAL,
          DFSConfigKeys.DFS_DISK_BALANCER_PLAN_VALID_INTERVAL_DEFAULT);
      if (planValidity.matches("[0-9]$")) {
        planValidity += "ms";
      }
      String errorString = "Plan was generated more than " + planValidity
          + " ago";
      LOG.error("Disk Balancer - " + errorString);
      throw new DiskBalancerException(errorString,
          DiskBalancerException.Result.OLD_PLAN_SUBMITTED);
    }
  }

  /**
   * Verify Node UUID.
   *
   * @param plan - Node Plan
   */
  private void verifyNodeUUID(NodePlan plan) throws DiskBalancerException {
    if ((plan.getNodeUUID() == null) ||
        !plan.getNodeUUID().equals(this.dataNodeUUID)) {
      LOG.error("Disk Balancer - Plan was generated for another node.");
      throw new DiskBalancerException(
          "Plan was generated for another node.",
          DiskBalancerException.Result.DATANODE_ID_MISMATCH);
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Regenerate the plan immediately before executing: `hdfs diskbalancer -plan <host:port>` followed by `hdfs diskbalancer -execute <planfile>`.
2. If the delay is intentional, raise dfs.disk.balancer.plan.valid.interval (e.g., '7d') in hdfs-site.xml on the DataNode and restart it.
3. Programmatic submitters that accept staleness can pass skipDateCheck=true to DiskBalancer.submitPlan.
4. Verify NTP on the plan-generation host and the DataNode to rule out clock skew.

Example fix

// before: submit a day-old plan with the date check enabled
diskBalancer.submitPlan(planID, version, planString, false);

// after: check age client-side and regenerate when stale
NodePlan p = NodePlan.parseJson(planString);
long validityMs = TimeUnit.DAYS.toMillis(2); // mirror dfs.disk.balancer.plan.valid.interval
if (p.getTimeStamp() + validityMs < Time.now()) {
  planString = generateFreshPlan(datanode);
}
diskBalancer.submitPlan(planID, version, planString, false);

long validityMs = TimeUnit.DAYS.toMillis(2); // mirror dfs.disk.balancer.plan.valid.interval
NodePlan p = NodePlan.parseJson(planString);
if (p.getTimeStamp() + validityMs < Time.now()) {
  // plan is stale: regenerate before submitting
}

try {
  diskBalancer.submitPlan(planID, version, plan, false);
} catch (DiskBalancerException e) {
  if (e.getResult() == DiskBalancerException.Result.OLD_PLAN_SUBMITTED) {
    plan = regeneratePlan(datanode);
    diskBalancer.submitPlan(newPlanID, version, plan, false); // one resubmit with a fresh plan
  }
}

When it happens

Trigger: Calling submitPlan(planID, version, plan, skipDateCheck=false) when plan.getTimeStamp() + planValidityInterval < Time.now() — i.e., the plan file was generated more than the configured interval (default 48 hours) before submission.

Common situations: Generating a plan on Friday and executing it Monday; batch tooling that generates plans for many nodes and then submits them slowly; a deliberately tiny validity interval such as '1000' (interpreted as ms); clock skew between the plan-generation host and the DataNode making a fresh plan look old.

Related errors

• Not a valid Boolean value for {property}
• Incompatible node types: storageType={storageType} but Stora
• No services to connect, missing NameNode address.
• BlockPoolSliceStorage.recoverTransitionRead: attempt to load
• No block pool offer service for bpid={}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/591b2defc8ce57b0. Report an issue: GitHub.