apache/druid · error · IllegalStateException

Cannot set mergers for final level more than once

Error message

Cannot set mergers for final level more than once

What it means

setTotalMergersForUltimateLevel is write-once: if the ultimate-level merger count is already set (different from SuperSorter.UNKNOWN_TOTAL), calling it again throws 'Cannot set mergers for final level more than once'. The ultimate level's merger count equals the number of output partitions and is finalized once partitioning is known; re-setting indicates the partition-count computation ran twice or two components both attempt to register it.

Source

Thrown at processing/src/main/java/org/apache/druid/frame/processor/SuperSorterProgressTracker.java:146

          totalMergingLevels - 1
      );
    }
    if (totalMergingLevels != SuperSorter.UNKNOWN_LEVEL
        && level < totalMergingLevels - 1 // This condition is only present for levels excluding the ultimate level
        && levelToTotalBatches.containsKey(level)) {
      throw new ISE("Total mergers are already present for the level %d", level);
    }
    levelToTotalBatches.put(level, totalMergers);
  }

  /**
   * Sets the number of mergers in the ultimate level (number of mergers = number of output partitions).
   * Can only be set once
   */
  public synchronized void setTotalMergersForUltimateLevel(final long totalMergersForUltimateLevel)
  {
    if (this.totalMergersForUltimateLevel != SuperSorter.UNKNOWN_TOTAL) {
      throw new ISE("Cannot set mergers for final level more than once");
    }
    this.totalMergersForUltimateLevel = totalMergersForUltimateLevel;
  }

  /**
   * This method is designed to be called during the course of the sorting. The batches once merged for a particular
   * level can be marked as such through this.
   */
  public synchronized void addMergedBatchesForLevel(final int level, final long additionalMergedBatches)
  {
    if (totalMergingLevels != SuperSorter.UNKNOWN_LEVEL && level >= totalMergingLevels) {
      throw new ISE(
          "Cannot add merged batches for level %d. Valid levels range from 0 to %d",
          level,
          totalMergingLevels - 1
      );
    }
    levelToMergedBatches.compute(level, (l, mergedBatchesSoFar) -> mergedBatchesSoFar == null

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Guard the call: only invoke setTotalMergersForUltimateLevel when the current value is still SuperSorter.UNKNOWN_TOTAL.
2. Ensure only one code path resolves output partitions and registers the ultimate-level count.
3. Construct a fresh tracker when retrying or reinitializing a sort.
4. If the output partition count genuinely changed, that requires a new sort run with a new tracker — the count is immutable once set.

Example fix

// before
tracker.setTotalMergersForUltimateLevel(outputPartitions.size()); // may be called on every future completion
// after
if (tracker.getTotalMergersForUltimateLevel() == SuperSorter.UNKNOWN_TOTAL) {
  tracker.setTotalMergersForUltimateLevel(outputPartitions.size());
}

if (tracker.getTotalMergersForUltimateLevel() == SuperSorter.UNKNOWN_TOTAL) {
  tracker.setTotalMergersForUltimateLevel(outputPartitions.size());
}

try {
  tracker.setTotalMergersForUltimateLevel(n);
} catch (IllegalStateException e) {
  if (!e.getMessage().contains("more than once")) {
    throw e;
  } // already set — ignore duplicates
}

When it happens

Trigger: Calling setTotalMergersForUltimateLevel a second time — e.g., outputPartitionsFuture completing twice, both the direct-merger and the middle-merger code paths registering output partitions, or a retry re-running sorter initialization on the same tracker.

Common situations: Reused tracker fixtures in tests; sorters re-initialized after cancellation without replacing the tracker; concurrent completion paths in custom merge drivers that both resolve the output partition count.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Total mergers are already present for the level %d
• Invalid level %d
• Total merging levels already defined for the merge sort.
• Max level found in levelToMergedBatches is %d (0-indexed). C
• Max level found in levelToTotalBatches is %d (0-indexed). Ca

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/9b0267129496e802. Report an issue: GitHub.