apache/druid · error · IllegalStateException

Total mergers are already present for the level %d

Error message

Total mergers are already present for the level %d

What it means

For non-ultimate levels (level < totalMergingLevels-1), setTotalMergersForLevel is write-once: if levelToTotalBatches already holds a total for that level, a second call throws. The ultimate level is exempt because its count gets overridden by totalMergersForUltimateLevel. Repeated registration of the same level's merger count indicates duplicated scheduling logic.

Source

Thrown at processing/src/main/java/org/apache/druid/frame/processor/SuperSorterProgressTracker.java:134

   * Sets the total mergers for a level. Can be set only once, except for the ultimate level (if total levels are known)
   * because they get overridden by totalMergersForUltimateLevel
   */
  public synchronized void setTotalMergersForLevel(final int level, final long totalMergers)
  {
    if (level < 0) {
      throw new ISE("Unable to set %d total mergers for level %d. Level must be non-negative", totalMergers, level);
    }
    if (totalMergingLevels != SuperSorter.UNKNOWN_LEVEL && level >= totalMergingLevels) {
      throw new ISE(
          "Cannot set total mergers for level %d. Valid levels range from 0 to %d",
          level,
          totalMergingLevels - 1
      );
    }
    if (totalMergingLevels != SuperSorter.UNKNOWN_LEVEL
        && level < totalMergingLevels - 1 // This condition is only present for levels excluding the ultimate level
        && levelToTotalBatches.containsKey(level)) {
      throw new ISE("Total mergers are already present for the level %d", level);
    }
    levelToTotalBatches.put(level, totalMergers);
  }

  /**
   * Sets the number of mergers in the ultimate level (number of mergers = number of output partitions).
   * Can only be set once
   */
  public synchronized void setTotalMergersForUltimateLevel(final long totalMergersForUltimateLevel)
  {
    if (this.totalMergersForUltimateLevel != SuperSorter.UNKNOWN_TOTAL) {
      throw new ISE("Cannot set mergers for final level more than once");
    }
    this.totalMergersForUltimateLevel = totalMergersForUltimateLevel;
  }

  /**
   * This method is designed to be called during the course of the sorting. The batches once merged for a particular

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Guard the call with a membership check (levelToTotalBatches already has the level) or track registration on the caller side.
2. Make registration idempotent: skip levels whose totals were already set instead of recomputing and re-setting.
3. Use a fresh SuperSorterProgressTracker when re-driving a sort after failure.
4. For the ultimate level, prefer setTotalMergersForUltimateLevel, which is the intended override mechanism.

Example fix

// before
tracker.setTotalMergersForLevel(level, mergers); // runs each scheduling pass
// after
if (!tracker.hasTotalMergersForLevel(level)) {
  tracker.setTotalMergersForLevel(level, mergers);
}

// Idempotent registration
if (!tracker.getLevelToTotalBatches().containsKey(level)) {
  tracker.setTotalMergersForLevel(level, mergers);
}

try {
  tracker.setTotalMergersForLevel(level, mergers);
} catch (IllegalStateException e) {
  if (!e.getMessage().contains("already present")) {
    throw e;
  } // duplicate registration — safe to ignore if value unchanged
}

When it happens

Trigger: Calling setTotalMergersForLevel twice for the same non-ultimate level — e.g., both a direct-merger path and a middle-merger path register totals for level 0, or merge scheduling re-runs after a partial failure.

Common situations: Retry logic re-invoking the merger-registration block without checking prior state; shared trackers reused across sorter phases; tests that call the setter in setup and again in the test body.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Cannot set mergers for final level more than once
• Invalid level %d
• Total merging levels already defined for the merge sort.
• Max level found in levelToMergedBatches is %d (0-indexed). C
• Max level found in levelToTotalBatches is %d (0-indexed). Ca

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/406e83854adbef51. Report an issue: GitHub.