apache/hadoop · error · InvalidInputException

Job ResourceSkyline history is containing less job runs than

Error message

Job ResourceSkyline history is containing less job runs than ${minJobRuns}, please try again with valid Job ResourceSkyline history

What it means

Error "Job ResourceSkyline history is containing less job runs than ${minJobRuns}, please try again with valid Job ResourceSkyline history" thrown in apache/hadoop.

Source

Thrown at hadoop-tools/hadoop-resourceestimator/src/main/java/org/apache/hadoop/resourceestimator/solver/preprocess/SolverPreprocessor.java:213

    List<ResourceSkyline> resourceSkylines = new ArrayList<ResourceSkyline>();
    for (Map.Entry<RecurrenceId, List<ResourceSkyline>> entry : jobHistory
        .entrySet()) {
      // TODO: identify different jobs within the same pipeline
      // right now, we do prediction at the granularity of pipeline, i.e., we
      // will merge the
      // resource skylines of jobs within the same pipeline into one aggregated
      // resource skyline
      ResourceSkyline skylineAgg = null;
      skylineAgg = mergeSkyline(entry.getValue());
      resourceSkylines.add(skylineAgg);
    }
    int numJobs = resourceSkylines.size();
    if (numJobs < minJobRuns) {
      LOGGER.error(
          "Solver requires job resource skyline history for at least {} runs,"
              + " but it only receives history info for {}  runs.",
          minJobRuns, numJobs);
      throw new InvalidInputException("Job ResourceSkyline history",
          "containing less job runs" + " than " + minJobRuns);
    }

    return resourceSkylines;
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Collect more job runs (at least minJobRuns) into the ResourceSkyline history before running the solver, or lower minJobRuns.

When it happens

Trigger: The job resource skyline history contains fewer job runs than the configured minimum, so the estimator cannot produce a reliable estimate.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7f549d1be10bdb8a. Report an issue: GitHub.