apache/druid · error · IllegalStateException

Cannot add merged batches for level

Error message

Cannot add merged batches for level %d. Valid levels range from 0 to %d

What it means

SuperSorterProgressTracker tracks how many batches have been merged at each sorting level. addMergedBatchesForLevel validates that the level is below totalMergingLevels (when known); passing a level at or above that bound means the sorter's bookkeeping is inconsistent with its configured merging depth. It throws ISE because this should be impossible with a correctly constructed tracker.

Solutions

1. Check the sorting configuration (e.g. max merging levels in frame processor / SuperSorter options) so totalMergingLevels matches the actual number of merge levels used
2. Verify the caller's loop only iterates levels 0..totalMergingLevels-1
3. Reproduce with a minimal query and file a Druid bug if internal code triggers it; avoid calling this API from external code
4. If it appears after an upgrade, compare frame-processor configs with the prior version and adjust

Example fix

// before
tracker.addMergedBatchesForLevel(level, batches); // level computed beyond totalMergingLevels
// after
if (tracker.getTotalMergingLevels() != SuperSorter.UNKNOWN_LEVEL && level < tracker.getTotalMergingLevels()) {
  tracker.addMergedBatchesForLevel(level, batches);
}

if (totalMergingLevels != SuperSorter.UNKNOWN_LEVEL && level >= totalMergingLevels) { throw new IllegalArgumentException("level out of range: " + level); }

boolean isValidLevel(int level, int totalMergingLevels) { return totalMergingLevels == SuperSorter.UNKNOWN_LEVEL || (level >= 0 && level < totalMergingLevels); }

try { tracker.addMergedBatchesForLevel(level, n); } catch (IllegalStateException e) { log.error("level bookkeeping mismatch: {}", e.getMessage()); }

When it happens

Trigger: Calling addMergedBatchesForLevel with a level >= totalMergingLevels when totalMergingLevels is known (not UNKNOWN_LEVEL); usually from an internal frame-processor pipeline whose per-level merge loop exceeded the configured maxMergingLevels.

Common situations: Custom or patched sorter logic miscomputing level counts; misconfigured deep-storage/limit configurations that reduce totalMergingLevels below what the query plan expects; regressions after upgrading Druid where frame-processor defaults changed.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Cannot set mergers for final level more than once
• Cannot set total mergers for level
• Invalid level
• Max level found in levelToMergedBatches is
• Max level found in levelToTotalBatches is

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/59766bcfdc3efe2e. Report an issue: GitHub.

Appendix: source

Thrown at processing/src/main/java/org/apache/druid/frame/processor/SuperSorterProgressTracker.java:158

   * Sets the number of mergers in the ultimate level (number of mergers = number of output partitions).
   * Can only be set once
   */
  public synchronized void setTotalMergersForUltimateLevel(final long totalMergersForUltimateLevel)
  {
    if (this.totalMergersForUltimateLevel != SuperSorter.UNKNOWN_TOTAL) {
      throw new ISE("Cannot set mergers for final level more than once");
    }
    this.totalMergersForUltimateLevel = totalMergersForUltimateLevel;
  }

  /**
   * This method is designed to be called during the course of the sorting. The batches once merged for a particular
   * level can be marked as such through this.
   */
  public synchronized void addMergedBatchesForLevel(final int level, final long additionalMergedBatches)
  {
    if (totalMergingLevels != SuperSorter.UNKNOWN_LEVEL && level >= totalMergingLevels) {
      throw new ISE(
          "Cannot add merged batches for level %d. Valid levels range from 0 to %d",
          level,
          totalMergingLevels - 1
      );
    }
    levelToMergedBatches.compute(level, (l, mergedBatchesSoFar) -> mergedBatchesSoFar == null
                                                                   ? additionalMergedBatches
                                                                   : additionalMergedBatches + mergedBatchesSoFar);
  }

  /**
   * If the SuperSorter is trivially done without doing any work (for eg - empty input), the tracker can be marked as
   * trivially complete. Once a tracker is marked as complete, the snapshots will always report back the progress
   * digest as 1. Any modification to the state of the tracker (eg: calling setTotalMergersForLevel()) would proceed
   * as regular, but
   */
  public synchronized void markTriviallyComplete()
  {

View on GitHub (pinned to 9b90983fd2)