apache/druid · error · IllegalStateException

No shuffle for stage[%d]

Error message

No shuffle for stage[%d]

What it means

generatePartitionBoundariesForShuffle() computes shuffle partition boundaries from gathered cluster-by statistics and is only meaningful for stages with a GLOBAL_SORT shuffle spec. If the stage's shuffleSpec is null — the stage does not shuffle — ISE("No shuffle for stage[%d]") is thrown with the stage number.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/StageDefinition.java:401

  }

  /**
   * Generate partition boundaries for {@link ShuffleKind#GLOBAL_SORT} shuffles.
   *
   * @param collector     statistics collector, to be provided if {@link #mustGatherResultKeyStatistics()}
   * @param maxPartitions maximum number of partitions to generate. On the controller, this is the value of
   *                      {@link MultiStageQueryContext#getMaxPartitions(QueryContext)}. On workers, this method
   *                      is only used when the number of partitions is determined ahead of time by the
   *                      {@link ShuffleSpec}, so {@link Integer#MAX_VALUE} is typically provided for this parameter
   *                      out of convenience.
   */
  public Either<Long, ClusterByPartitions> generatePartitionBoundariesForShuffle(
      @Nullable ClusterByStatisticsCollector collector,
      int maxPartitions
  )
  {
    if (shuffleSpec == null) {
      throw new ISE("No shuffle for stage[%d]", getStageNumber());
    } else if (shuffleSpec.kind() != ShuffleKind.GLOBAL_SORT) {
      throw new ISE(
          "Shuffle of kind [%s] cannot generate partition boundaries for stage[%d]",
          shuffleSpec.kind(),
          getStageNumber()
      );
    } else if (mustGatherResultKeyStatistics() && collector == null) {
      throw new ISE("Statistics required, but not gathered for stage[%d]", getStageNumber());
    } else if (!mustGatherResultKeyStatistics() && collector != null) {
      throw new ISE("Statistics gathered, but not required for stage[%d]", getStageNumber());
    } else {
      return ((GlobalSortShuffleSpec) shuffleSpec).generatePartitionsForGlobalSort(collector, maxPartitions);
    }
  }

  public ClusterByStatisticsCollector createResultKeyStatisticsCollector(
      final FrameType frameType,
      final int maxRetainedBytes

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Only call generatePartitionBoundariesForShuffle on stages whose shuffle spec is non-null (check getShuffleSpecForDiagnostics or the definition)
2. Ensure the stage graph assigns a GLOBAL_SORT shuffle spec to stages that must produce partition boundaries
3. Fix the controller's stage-builder so leaf stages are not asked to generate boundaries

Example fix

// before
boundaries = stageDef.generatePartitionBoundariesForShuffle(collector, maxParts);
// after
if (stageDef.getProcessor() != null && stageDef.hasShuffle()) {
  boundaries = stageDef.generatePartitionBoundariesForShuffle(collector, maxParts);
}

if (stageDef.getProcessor() == null || /* no shuffle spec */ true) {
  throw new IllegalStateException("stage has no shuffle; skip boundary generation");
}

boolean canGenerateBoundaries(StageDefinition d) {
  try { return d.getShuffleSpecForDiagnostics() != null; } catch (Exception e) { return false; }
}

try { boundaries = d.generatePartitionBoundariesForShuffle(c, maxP); } catch (IllegalStateException e) { boundaries = Either.value(0L); }

When it happens

Trigger: WorkerStageKernel calling generatePartitionBoundariesForShuffle(...) on a stage whose definition has no shuffle spec (leaf/source stage), or test code invoking it directly on a non-shuffling StageDefinition.

Common situations: Custom worker logic or tooling that assumes every stage needs partition boundaries; unit tests reusing a builder without setting a shuffle spec.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Shuffle of kind [%s] cannot generate partition boundaries fo
• Statistics required, but not gathered for stage[%d]
• Statistics gathered, but not required for stage[%d]
• Result partition information is not relevant to this stage b
• Root partition range[%d, %d] of new segments doesn't match t

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/ef10cdff9f196e34. Report an issue: GitHub.