apache/druid · error · IllegalStateException

Cannot set total mergers for level %d. Valid levels range fr

Error message

Cannot set total mergers for level %d. Valid levels range from 0 to %d

What it means

setTotalMergersForLevel throws this when the total merging level count is already known and the given level is at or beyond it. Once totalMergingLevels is fixed, only levels 0..totalMergingLevels-1 are valid targets for merger totals; anything else means the caller's level numbering no longer matches the defined hierarchy.

Source

Thrown at processing/src/main/java/org/apache/druid/frame/processor/SuperSorterProgressTracker.java:125

            totalMergingLevels
        );
      }
    });

    this.totalMergingLevels = totalMergingLevels;
  }

  /**
   * Sets the total mergers for a level. Can be set only once, except for the ultimate level (if total levels are known)
   * because they get overridden by totalMergersForUltimateLevel
   */
  public synchronized void setTotalMergersForLevel(final int level, final long totalMergers)
  {
    if (level < 0) {
      throw new ISE("Unable to set %d total mergers for level %d. Level must be non-negative", totalMergers, level);
    }
    if (totalMergingLevels != SuperSorter.UNKNOWN_LEVEL && level >= totalMergingLevels) {
      throw new ISE(
          "Cannot set total mergers for level %d. Valid levels range from 0 to %d",
          level,
          totalMergingLevels - 1
      );
    }
    if (totalMergingLevels != SuperSorter.UNKNOWN_LEVEL
        && level < totalMergingLevels - 1 // This condition is only present for levels excluding the ultimate level
        && levelToTotalBatches.containsKey(level)) {
      throw new ISE("Total mergers are already present for the level %d", level);
    }
    levelToTotalBatches.put(level, totalMergers);
  }

  /**
   * Sets the number of mergers in the ultimate level (number of mergers = number of output partitions).
   * Can only be set once
   */
  public synchronized void setTotalMergersForUltimateLevel(final long totalMergersForUltimateLevel)

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Register the ultimate-level merger count via setTotalMergersForUltimateLevel instead of setTotalMergersForLevel for level == totalMergingLevels.
2. Validate level < totalMergingLevels at the call site before setting.
3. Recompute totalMergingLevels so it covers all levels the scheduler will touch, and set it before scheduling.
4. Check off-by-one comparisons against totalMergingLevels - 1 in the merge driver.

Example fix

// before
tracker.setTotalMergersForLevel(totalMergingLevels, outputPartitions); // ultimate level
// after
tracker.setTotalMergersForUltimateLevel(outputPartitions);

int total = tracker.getTotalMergingLevels();
if (total == SuperSorter.UNKNOWN_LEVEL || level < total) {
  tracker.setTotalMergersForLevel(level, mergers);
}

boolean isRegularLevel(int level, int totalMergingLevels) {
  return totalMergingLevels != SuperSorter.UNKNOWN_LEVEL && level >= 0 && level < totalMergingLevels - 1;
}

When it happens

Trigger: Calling setTotalMergersForLevel(level, mergers) with level >= totalMergingLevels after setTotalMergingLevels ran; e.g., the ultimate level's count is being registered as a regular level, or the level count was reduced after merger registration started.

Common situations: Off-by-one confusion around the ultimate level (totalMergingLevels-1) in custom sorter code; dynamic partition changes shrinking levels while merge scheduling continues with old indices.

Related errors

• Invalid level %d
• Max level found in levelToMergedBatches is %d (0-indexed). C
• Max level found in levelToTotalBatches is %d (0-indexed). Ca
• Unable to set %d total mergers for level %d. Level must be n
• Total mergers are already present for the level %d

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/cf990561cfb30827. Report an issue: GitHub.