{"record":{"id":"4a2a55dcdaec40e6","repo":"apache/druid","slug":"output-partitions-are-not-ready-yet","errorCode":null,"errorMessage":"Output partitions are not ready yet","messagePattern":"Output partitions are not ready yet","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/frame/processor/SuperSorter.java","lineNumber":908,"sourceCode":"      } else {\n        // Use two levels: no need to have a partitioned penultimate layer.\n        totalMergingLevels = 2;\n      }\n    } else {\n      totalMergingLevels = level + 1;\n    }\n\n    for (int i = level; i < totalMergingLevels; i++) {\n      superSorterProgressTracker.setTotalMergersForLevel(i, 1);\n    }\n\n    superSorterProgressTracker.setTotalMergingLevels(totalMergingLevels);\n  }\n\n  private ClusterByPartitions getOutputPartitions()\n  {\n    if (!outputPartitionsFuture.isDone()) {\n      throw new ISE(\"Output partitions are not ready yet\");\n    }\n\n    return FutureUtils.getUnchecked(outputPartitionsFuture, true);\n  }\n\n  @GuardedBy(\"runWorkersLock\")\n  private long getTotalMergersInLevel(final int level)\n  {\n    if (totalInputFrames == UNKNOWN_TOTAL || totalMergingLevels == UNKNOWN_LEVEL) {\n      return UNKNOWN_TOTAL;\n    } else if (level >= totalMergingLevels) {\n      throw new ISE(\"Invalid level %d\", level);\n    } else if (level == totalMergingLevels - 1) {\n      if (outputPartitionsFuture.isDone()) {\n        return totalInputFrames == 0 ? 0 : getOutputPartitions().size();\n      } else {\n        return UNKNOWN_TOTAL;\n      }","sourceCodeStart":890,"sourceCodeEnd":926,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/frame/processor/SuperSorter.java#L890-L926","documentation":"getOutputPartitions() reads the partitioning scheme, but only after it has been computed by the partitioning phase. If outputPartitionsFuture is not yet done, it throws IllegalStateException — callers must not read output partitions before the partition-discovery step of the sort completes. It is an internal timing/contract guard: waiting should be done via the future, not by polling this method.","triggerScenarios":"Calling getOutputPartitions() (directly or indirectly via partitions(), outputPartitionCount(), or runNextUltimateMerger()) while outputPartitionsFuture is still pending — e.g. invoking outputPartitionCount() right after run() before the partition-determining processors finish, or wiring an ultimate merger before partition info resolves.","commonSituations":"Custom code extending or instrumenting SuperSorter that reads partition info too early; debugging/instrumentation logging partition counts immediately after starting the sorter; race conditions in modified scheduling code that skips the future-completion callback that normally triggers these reads.","solutions":["Await outputPartitionsFuture (or the future returned by run()) before reading output partitions; do not poll getOutputPartitions().","Hook the completion callback: read partitions only after the partition-determination phase future completes (as run() does via its listener).","If this occurs inside stock Druid code paths, verify no custom extension altered the ordering of runMerger/setTotalMergingLevelsIfPossible calls.","Upgrade Druid if hit spontaneously — this can indicate an internal scheduling regression fixed in later releases."],"exampleFix":"// before\nint partitions = superSorter.outputPartitionCount(); // may throw if partitions unknown yet\n// after\nListenableFuture<ClusterByPartitions> partitionsFuture = superSorter.outputPartitionsFuture();\nClusterByPartitions partitions = FutureUtils.getUnchecked(partitionsFuture, true);","handlingStrategy":"validation","validationCode":"if (!outputPartitionsFuture.isDone()) {\n  // wait instead of reading\n  ClusterByPartitions partitions = FutureUtils.getUnchecked(outputPartitionsFuture, true);\n}","typeGuard":"static boolean outputPartitionsReady(ListenableFuture<ClusterByPartitions> f) {\n  return f.isDone();\n}","tryCatchPattern":"try {\n  partitions = superSorter.outputPartitionCount();\n} catch (IllegalStateException e) {\n  partitions = FutureUtils.getUnchecked(partitionsFuture, true).size();\n}","preventionTips":["Always await outputPartitionsFuture before querying partition info.","Trigger dependent work from future-completion callbacks, not eager polling.","Do not instrument or extend SuperSorter in ways that reorder merger startup.","If spontaneous, check for Druid version regressions in sorter scheduling."],"tags":["state","concurrency","timing"],"backgroundTag":"invalid-state-transition","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}