apache/druid · error · IllegalStateException

Shuffle of kind [%s] cannot generate partition boundaries fo

Error message

Shuffle of kind [%s] cannot generate partition boundaries for stage[%d]

What it means

generatePartitionBoundariesForShuffle() only supports shuffle specs of kind GLOBAL_SORT, because partition boundaries are derived from global sort-key statistics. If shuffleSpec.kind() is any other ShuffleKind (e.g. HASH or BROADCAST), ISE("Shuffle of kind [%s] cannot generate partition boundaries for stage[%d]") is thrown.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/StageDefinition.java:403

  /**
   * Generate partition boundaries for {@link ShuffleKind#GLOBAL_SORT} shuffles.
   *
   * @param collector     statistics collector, to be provided if {@link #mustGatherResultKeyStatistics()}
   * @param maxPartitions maximum number of partitions to generate. On the controller, this is the value of
   *                      {@link MultiStageQueryContext#getMaxPartitions(QueryContext)}. On workers, this method
   *                      is only used when the number of partitions is determined ahead of time by the
   *                      {@link ShuffleSpec}, so {@link Integer#MAX_VALUE} is typically provided for this parameter
   *                      out of convenience.
   */
  public Either<Long, ClusterByPartitions> generatePartitionBoundariesForShuffle(
      @Nullable ClusterByStatisticsCollector collector,
      int maxPartitions
  )
  {
    if (shuffleSpec == null) {
      throw new ISE("No shuffle for stage[%d]", getStageNumber());
    } else if (shuffleSpec.kind() != ShuffleKind.GLOBAL_SORT) {
      throw new ISE(
          "Shuffle of kind [%s] cannot generate partition boundaries for stage[%d]",
          shuffleSpec.kind(),
          getStageNumber()
      );
    } else if (mustGatherResultKeyStatistics() && collector == null) {
      throw new ISE("Statistics required, but not gathered for stage[%d]", getStageNumber());
    } else if (!mustGatherResultKeyStatistics() && collector != null) {
      throw new ISE("Statistics gathered, but not required for stage[%d]", getStageNumber());
    } else {
      return ((GlobalSortShuffleSpec) shuffleSpec).generatePartitionsForGlobalSort(collector, maxPartitions);
    }
  }

  public ClusterByStatisticsCollector createResultKeyStatisticsCollector(
      final FrameType frameType,
      final int maxRetainedBytes
  )
  {

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Check shuffleSpec.kind() == ShuffleKind.GLOBAL_SORT before calling; skip boundary generation for hash/broadcast shuffles
2. If global partitioning is required, replace the shuffle spec with GlobalSortShuffleSpec
3. Route hash-shuffled stages through a different code path that does not need cluster-by statistics

Example fix

// before
boundaries = stageDef.generatePartitionBoundariesForShuffle(collector, maxParts);
// after
if (stageDef.getShuffleSpecForDiagnostics().kind() == ShuffleKind.GLOBAL_SORT) {
  boundaries = stageDef.generatePartitionBoundariesForShuffle(collector, maxParts);
}

if (stageDef.getShuffleSpecForDiagnostics().kind() != ShuffleKind.GLOBAL_SORT) {
  throw new IllegalStateException("boundary generation requires GLOBAL_SORT shuffle");
}

boolean isGlobalSort(StageDefinition d) {
  return d.getShuffleSpecForDiagnostics().kind() == ShuffleKind.GLOBAL_SORT;
}

try { boundaries = d.generatePartitionBoundariesForShuffle(c, maxP); } catch (IllegalStateException e) { /* route hash shuffles elsewhere */ }

When it happens

Trigger: Calling generatePartitionBoundariesForShuffle() on a stage whose shuffle spec is a hash/broadcast shuffle rather than GlobalSortShuffleSpec.

Common situations: Custom query tooling that applies boundary generation uniformly to all shuffle stages, or a query plan mixing hash shuffles with global-sort boundary logic.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Number of partitions not known for [%s].
• No shuffle for stage[%d]
• Statistics required, but not gathered for stage[%d]
• Statistics gathered, but not required for stage[%d]
• Result partition information is not relevant to this stage b

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/ffd760fac21e176f. Report an issue: GitHub.