apache/druid · error · IllegalStateException

Invalid level %d

Error message

Invalid level %d

What it means

SuperSorter.getTotalMergersInLevel throws this IllegalStateException when asked for the merger count of a level that is greater than or equal to the total number of merging levels. Valid levels are 0 to totalMergingLevels-1; requesting anything higher means a caller is indexing beyond the defined sort hierarchy. This is an internal consistency check within the frame-processor SuperSorter machinery, not an expected user-facing failure.

Source

Thrown at processing/src/main/java/org/apache/druid/frame/processor/SuperSorter.java:920

    superSorterProgressTracker.setTotalMergingLevels(totalMergingLevels);
  }

  private ClusterByPartitions getOutputPartitions()
  {
    if (!outputPartitionsFuture.isDone()) {
      throw new ISE("Output partitions are not ready yet");
    }

    return FutureUtils.getUnchecked(outputPartitionsFuture, true);
  }

  @GuardedBy("runWorkersLock")
  private long getTotalMergersInLevel(final int level)
  {
    if (totalInputFrames == UNKNOWN_TOTAL || totalMergingLevels == UNKNOWN_LEVEL) {
      return UNKNOWN_TOTAL;
    } else if (level >= totalMergingLevels) {
      throw new ISE("Invalid level %d", level);
    } else if (level == totalMergingLevels - 1) {
      if (outputPartitionsFuture.isDone()) {
        return totalInputFrames == 0 ? 0 : getOutputPartitions().size();
      } else {
        return UNKNOWN_TOTAL;
      }
    } else if (level > 0 && level == totalMergingLevels - 2) {
      if (outputPartitionsFuture.isDone()) {
        // Smallest number of mergers we can possibly use in the penultimate level.
        final long totalInputs = getTotalMergersInLevel(level - 1);
        final long minMergers =
            LongMath.divide(totalInputs, maxChannelsPerMerger, RoundingMode.CEILING);

        // Ensure we have a maximal degree of parallelism: possibly use more mergers than minMergers.
        long targetNumMergers = Math.max(
            minMergers,
            Math.min(
                maxActiveProcessors,

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Verify totalMergingLevels was set (via SuperSorterProgressTracker.setTotalMergingLevels) before any level-indexed merge scheduling begins.
2. Check that all level arithmetic is exclusive of the total: valid levels are 0..totalMergingLevels-1; fix any off-by-one comparisons such as level <= totalMergingLevels.
3. Ensure callers handle the UNKNOWN_LEVEL sentinel instead of passing it as a concrete level value.
4. If the exception is reproducible with a fixed query shape, capture the stateString() log output and report the bug with the query's partitioning/sorting configuration.

Example fix

// before
for (int level = 0; level <= totalMergingLevels; level++) {
  long mergers = getTotalMergersInLevel(level);
}
// after
for (int level = 0; level < totalMergingLevels; level++) {
  long mergers = getTotalMergersInLevel(level);
}

if (totalMergingLevels != SuperSorter.UNKNOWN_LEVEL && level >= 0 && level < totalMergingLevels) {
  long mergers = getTotalMergersInLevel(level);
}

boolean isValidLevel(int level, int totalMergingLevels) {
  return level >= 0 && totalMergingLevels != SuperSorter.UNKNOWN_LEVEL && level < totalMergingLevels;
}

try {
  mergers = getTotalMergersInLevel(level);
} catch (IllegalStateException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Invalid level")) {
    mergers = SuperSorter.UNKNOWN_TOTAL; // treat as unknown rather than failing the query
  } else {
    throw e;
  }
}

When it happens

Trigger: Calling getTotalMergersInLevel(level) where level >= totalMergingLevels, or when totalMergingLevels logic in one of its callers (setAllDoneIfPossible, runNextDirectMerger, totalInputs, runNextMiddleMerger, runNextUltimateMerger) computes an off-by-one or uninitialized level value.

Common situations: Developers modifying SuperSorter or writing custom frame processors that drive the sorter pass a level derived from their own bookkeeping that no longer matches the sorter's level count; also seen when totalInputFrames/totalMergingLevels were set after merge scheduling already started, leading to stale level indices.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Cannot set total mergers for level %d. Valid levels range fr
• Max level found in levelToMergedBatches is %d (0-indexed). C
• Max level found in levelToTotalBatches is %d (0-indexed). Ca
• Unable to set %d total mergers for level %d. Level must be n
• Total mergers are already present for the level %d

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/0daf0db400c31ed5. Report an issue: GitHub.