apache/druid · error · IllegalStateException

Max level found in levelToTotalBatches is %d (0-indexed). Ca

Error message

Max level found in levelToTotalBatches is %d (0-indexed). Cannot set totalMergingLevels to %d

What it means

Same validation as the levelToMergedBatches check, but for levelToTotalBatches: if the tracker's recorded total-batch levels already reach or exceed the proposed totalMergingLevels, setTotalMergingLevels throws. This guarantees the write-once level count stays consistent with all existing per-level batch totals (0-indexed levels).

Source

Thrown at processing/src/main/java/org/apache/druid/frame/processor/SuperSorterProgressTracker.java:104

   * Set total merging levels for the SuperSorter it is tracking. Can be set only once
   */
  public synchronized void setTotalMergingLevels(final int totalMergingLevels)
  {
    if (this.totalMergingLevels != SuperSorter.UNKNOWN_LEVEL) {
      throw new ISE("Total merging levels already defined for the merge sort.");
    }
    levelToMergedBatches.keySet().stream().max(Ordering.natural()).ifPresent(max -> {
      if (max >= totalMergingLevels) {
        throw new ISE(
            "Max level found in levelToMergedBatches is %d (0-indexed). Cannot set totalMergingLevels to %d",
            max,
            totalMergingLevels
        );
      }
    });
    levelToTotalBatches.keySet().stream().max(Ordering.natural()).ifPresent(max -> {
      if (max >= totalMergingLevels) {
        throw new ISE(
            "Max level found in levelToTotalBatches is %d (0-indexed). Cannot set totalMergingLevels to %d",
            max,
            totalMergingLevels
        );
      }
    });

    this.totalMergingLevels = totalMergingLevels;
  }

  /**
   * Sets the total mergers for a level. Can be set only once, except for the ultimate level (if total levels are known)
   * because they get overridden by totalMergersForUltimateLevel
   */
  public synchronized void setTotalMergersForLevel(final int level, final long totalMergers)
  {
    if (level < 0) {
      throw new ISE("Unable to set %d total mergers for level %d. Level must be non-negative", totalMergers, level);

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Call setTotalMergingLevels before any setTotalMergersForLevel calls so levelToTotalBatches is empty when levels are set.
2. Recompute totalMergingLevels to cover every level for which mergers were already registered.
3. Replace the tracker with a new instance if the level structure of the sort has changed mid-run.
4. Add an assertion in test code that totalMergingLevels is set first, to catch ordering regressions early.

Example fix

// before
tracker.setTotalMergersForLevel(2, 4);
tracker.setTotalMergingLevels(2); // valid levels 0..1 -> throws
// after
tracker.setTotalMergingLevels(3); // covers levels 0..2
tracker.setTotalMergersForLevel(2, 4);

assert tracker.getLevelToTotalBatches().keySet().stream().max(Integer::compare).orElse(-1) < levels
    : "existing level totals exceed new totalMergingLevels";

When it happens

Trigger: setTotalMergersForLevel was called (populating levelToTotalBatches) for a level at or above the value later passed to setTotalMergingLevels — i.e., level totals were registered before the level count was finalized.

Common situations: See trigger scenarios.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Max level found in levelToMergedBatches is %d (0-indexed). C
• Invalid level %d
• Unable to set %d total mergers for level %d. Level must be n
• Cannot set total mergers for level %d. Valid levels range fr
• Total mergers are already present for the level %d

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/796bc232655d39f9. Report an issue: GitHub.