apache/hadoop · error · DuplicateRecurrenceIdException

Trying to addHistory duplicate resource skylines for {}. Use

Error message

Trying to addHistory duplicate resource skylines for {}. Use updateHistory function instead.

What it means

Error "Trying to addHistory duplicate resource skylines for {}. Use updateHistory function instead." thrown in apache/hadoop.

Source

Thrown at hadoop-tools/hadoop-resourceestimator/src/main/java/org/apache/hadoop/resourceestimator/skylinestore/impl/InMemoryStore.java:105

          // if filteredInput has duplicate jobIds with existing skylines in the
          // store,
          // throw out an exception
          final List<ResourceSkyline> jobHistory =
              skylineStore.get(recurrenceId);
          final List<String> oldJobIds = new ArrayList<>();
          for (final ResourceSkyline resourceSkyline : jobHistory) {
            oldJobIds.add(resourceSkyline.getJobId());
          }
          if (!oldJobIds.isEmpty()) {
            for (ResourceSkyline elem : filteredInput) {
              if (oldJobIds.contains(elem.getJobId())) {
                StringBuilder errMsg = new StringBuilder();
                errMsg.append(
                    "Trying to addHistory duplicate resource skylines for "
                        + recurrenceId
                        + ". Use updateHistory function instead.");
                LOGGER.error(errMsg.toString());
                throw new DuplicateRecurrenceIdException(errMsg.toString());
              }
            }
          }
          skylineStore.get(recurrenceId).addAll(filteredInput);
          LOGGER.info("Successfully addHistory new resource skylines for {}.",
              recurrenceId);
        } else {
          skylineStore.put(recurrenceId, filteredInput);
          LOGGER.info("Successfully addHistory new resource skylines for {}.",
              recurrenceId);
        }
      }
    } finally {
      writeLock.unlock();
    }
  }

  @Override public void addEstimation(String pipelineId,

View on GitHub (pinned to 2add963021)

Solutions

1. Call updateHistory instead of addHistory for a recurrence id that already has resource skylines.

When it happens

Trigger: addHistory is called for a recurrence id that already has resource skylines in the InMemorySkylineStore; callers must use updateHistory for existing pipelines.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/cf272ee19baee200. Report an issue: GitHub.