apache/druid · error · IllegalStateException

No statistics needed for stage[%d]

Error message

No statistics needed for stage[%d]

What it means

createResultKeyStatisticsCollector() builds the ClusterByStatisticsCollector used to track shuffle key distributions. Calling it on a stage whose mustGatherResultKeyStatistics() is false is a programming error — no statistics exist to collect — so ISE("No statistics needed for stage[%d]") is thrown.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/StageDefinition.java:423

          shuffleSpec.kind(),
          getStageNumber()
      );
    } else if (mustGatherResultKeyStatistics() && collector == null) {
      throw new ISE("Statistics required, but not gathered for stage[%d]", getStageNumber());
    } else if (!mustGatherResultKeyStatistics() && collector != null) {
      throw new ISE("Statistics gathered, but not required for stage[%d]", getStageNumber());
    } else {
      return ((GlobalSortShuffleSpec) shuffleSpec).generatePartitionsForGlobalSort(collector, maxPartitions);
    }
  }

  public ClusterByStatisticsCollector createResultKeyStatisticsCollector(
      final FrameType frameType,
      final int maxRetainedBytes
  )
  {
    if (!mustGatherResultKeyStatistics()) {
      throw new ISE("No statistics needed for stage[%d]", getStageNumber());
    }

    return ClusterByStatisticsCollectorImpl.create(
        shuffleSpec.clusterBy(),
        signature,
        frameType,
        maxRetainedBytes,
        Limits.MAX_PARTITION_BUCKETS,
        ((GlobalSortShuffleSpec) shuffleSpec).doesAggregate(),
        shuffleCheckHasMultipleValues
    );
  }

  /**
   * Create the {@link FrameWriterFactory} that must be used by {@link #getProcessor()}.
   *
   * Calls {@link MemoryAllocatorFactory#newAllocator()} for each frame.
   */

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Check stageDefinition.mustGatherResultKeyStatistics() before calling createResultKeyStatisticsCollector
2. Skip statistics collection for stages whose shuffle spec is not GLOBAL_SORT
3. If statistics are expected, correct the shuffle spec assigned to the stage

Example fix

// before
ClusterByStatisticsCollector c = stageDef.createResultKeyStatisticsCollector(frameType, maxBytes);
// after
if (stageDef.mustGatherResultKeyStatistics()) {
  ClusterByStatisticsCollector c = stageDef.createResultKeyStatisticsCollector(frameType, maxBytes);
}

if (!stageDef.mustGatherResultKeyStatistics()) {
  return; // skip collector creation
}

boolean needsStats(StageDefinition d) { return d.mustGatherResultKeyStatistics(); }

try { c = d.createResultKeyStatisticsCollector(ft, bytes); } catch (IllegalStateException e) { c = null; }

When it happens

Trigger: Calling createResultKeyStatisticsCollector(frameType, maxRetainedBytes) on a non-global-sort stage (shuffleSpec null or kind != GLOBAL_SORT), e.g. from gatherResultKeyStatistics or worker code iterating all stages.

Common situations: Worker code that unconditionally creates a statistics collector for every stage, or tooling that assumes all shuffle stages gather statistics.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Cannot shuffle with spec [%s] and nil clusterBy
• Stage does not shuffle
• Statistics required, but not gathered for stage[%d]
• Statistics gathered, but not required for stage[%d]
• Unknown

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/d9cac610cbe36a42. Report an issue: GitHub.