{"record":{"id":"6bad13229d0a35c4","repo":"apache/druid","slug":"max-level-found-in-leveltomergedbatches-is-d-0-i","errorCode":null,"errorMessage":"Max level found in levelToMergedBatches is %d (0-indexed). Cannot set totalMergingLevels to %d","messagePattern":"Max level found in levelToMergedBatches is (.+?) \\(0-indexed\\)\\. Cannot set totalMergingLevels to (.+?)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/frame/processor/SuperSorterProgressTracker.java","lineNumber":95,"sourceCode":"  );\n\n  public SuperSorterProgressTracker()\n  {\n    this.levelToMergedBatches = new HashMap<>();\n    this.levelToTotalBatches = new HashMap<>();\n  }\n\n  /**\n   * Set total merging levels for the SuperSorter it is tracking. Can be set only once\n   */\n  public synchronized void setTotalMergingLevels(final int totalMergingLevels)\n  {\n    if (this.totalMergingLevels != SuperSorter.UNKNOWN_LEVEL) {\n      throw new ISE(\"Total merging levels already defined for the merge sort.\");\n    }\n    levelToMergedBatches.keySet().stream().max(Ordering.natural()).ifPresent(max -> {\n      if (max >= totalMergingLevels) {\n        throw new ISE(\n            \"Max level found in levelToMergedBatches is %d (0-indexed). Cannot set totalMergingLevels to %d\",\n            max,\n            totalMergingLevels\n        );\n      }\n    });\n    levelToTotalBatches.keySet().stream().max(Ordering.natural()).ifPresent(max -> {\n      if (max >= totalMergingLevels) {\n        throw new ISE(\n            \"Max level found in levelToTotalBatches is %d (0-indexed). Cannot set totalMergingLevels to %d\",\n            max,\n            totalMergingLevels\n        );\n      }\n    });\n\n    this.totalMergingLevels = totalMergingLevels;\n  }","sourceCodeStart":77,"sourceCodeEnd":113,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/frame/processor/SuperSorterProgressTracker.java#L77-L113","documentation":"In setTotalMergingLevels, after confirming the level count is unset, the tracker validates existing batch bookkeeping: if levelToMergedBatches already contains a level index >= the proposed totalMergingLevels, setting the count would invalidate those records, so it throws. The recorded merged-batch levels must fit inside the new total level range (0-indexed).","triggerScenarios":"Progress was already tracked (updateMergedBatches) with level indices derived from an assumed or previous level count, then setTotalMergingLevels is called with a smaller value; typically occurs when merge work started before the level count was finalized.","commonSituations":"A sorter whose partition count shrinks between planning and execution (dynamic partition changes), causing previously recorded levels to exceed the new total; or unit tests that pre-populate batch progress before setting levels.","solutions":["Always call setTotalMergingLevels before recording any merged batches so levelToMergedBatches is empty at set time.","Ensure the computed totalMergingLevels accounts for the maximum level actually in use by the merge scheduler.","If partition/level counts changed, create a fresh tracker rather than mutating the existing one.","Audit callers that update progress with level indices and confirm they derive the index from the same totalMergingLevels value."],"exampleFix":"// before\ntracker.updateMergedBatches(level, batches); // level from old plan\ntracker.setTotalMergingLevels(newSmallerLevels); // throws\n// after\ntracker.setTotalMergingLevels(levels); // set first\ntracker.updateMergedBatches(level, batches);","handlingStrategy":"validation","validationCode":"// Ensure no merged-batch records exist before setting levels\nassert tracker.getLevelToMergedBatches().isEmpty() : \"set totalMergingLevels before tracking batches\";","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Set totalMergingLevels as the first tracker operation.","Never record merged batches before the level count is finalized.","Rebuild the tracker when partition/level structure changes mid-run."],"tags":["internal-state","initialization-order","sorting"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}