apache/druid · error · IllegalStateException

Number of partitions not known for [%s].

Error message

Number of partitions not known for [%s].

What it means

For target-size shuffles, the exact number of partitions cannot be known until the global sort has processed the cluster-by statistics; partitions are generated dynamically via generatePartitionsForGlobalSort. partitionCount() therefore always throws ISE for this spec — it is intentionally unsupported.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/GlobalSortTargetSizeShuffleSpec.java:88

  @Override
  @JsonProperty("aggregate")
  @JsonInclude(JsonInclude.Include.NON_DEFAULT)
  public boolean doesAggregate()
  {
    return aggregate;
  }

  @Override
  public boolean mustGatherResultKeyStatistics()
  {
    return true;
  }

  @Override
  public int partitionCount()
  {
    throw new ISE("Number of partitions not known for [%s].", kind());
  }

  @Override
  public Either<Long, ClusterByPartitions> generatePartitionsForGlobalSort(
      @Nullable final ClusterByStatisticsCollector collector,
      final int maxNumPartitions
  )
  {
    final long expectedPartitions = collector.estimatedTotalWeight() / targetSize;

    if (expectedPartitions > maxNumPartitions) {
      return Either.error(expectedPartitions);
    } else {
      collector.logSketches();
      final ClusterByPartitions generatedPartitions = collector.generatePartitionsWithTargetWeight(targetSize);
      if (generatedPartitions.size() <= maxNumPartitions) {
        return Either.value(generatedPartitions);
      } else {

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Do not call partitionCount() on target-size specs; use generatePartitionsForGlobalSort(collector, maxNumPartitions) after collecting statistics
2. Check the shuffle kind with instanceof/kind() before calling partitionCount()
3. Use the targetSize configuration to reason about expected partition count rather than partitionCount()

Example fix

// before
int partitions = shuffleSpec.partitionCount();
// after
if (shuffleSpec instanceof GlobalSortTargetSizeShuffleSpec) {
  Either<Long, ClusterByPartitions> p = shuffleSpec.generatePartitionsForGlobalSort(collector, maxNumPartitions);
} else {
  int partitions = shuffleSpec.partitionCount();
}

if (shuffleSpec instanceof GlobalSortTargetSizeShuffleSpec) {
  // partitionCount() is always unsupported; use generatePartitionsForGlobalSort
}

static boolean hasStaticPartitionCount(ShuffleSpec spec) {
  return !(spec instanceof GlobalSortTargetSizeShuffleSpec);
}

try {
  return spec.partitionCount();
} catch (IllegalStateException e) {
  return computeFromTargetSize(spec); // derive dynamically
}

When it happens

Trigger: Any call to partitionCount() on a GlobalSortTargetSizeShuffleSpec instance, regardless of configuration (e.g. framework code that reports partition counts before execution).

Common situations: Code that uniformly asks shuffle specs for a static partition count; mixing specs where some (MAX_COUNT with maxPartitions=1) can answer and target-size cannot.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Partition count must be at least 1
• Partition count must be 1 when adjustable is true, but was [
• ClusterBy key must be sortable
• Cannot bucket with %s partitioning
• Number of partitions not known for [%s] with maxPartitions[%

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/40b615d46891eeee. Report an issue: GitHub.