apache/druid · error · IllegalStateException

Number of partitions not known for [%s] with maxPartitions[%

Error message

Number of partitions not known for [%s] with maxPartitions[%d].

What it means

GlobalSortMaxCountShuffleSpec.partitionCount() can only return a definitive count when maxPartitions == 1. For any other maxPartitions the actual number of partitions is determined only after stage execution, so calling partitionCount() throws ISE. Callers must use generatePartitionsForGlobalSort or maxPartitions instead.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/GlobalSortMaxCountShuffleSpec.java:146

        return Either.error((long) generatedPartitions.size());
      }
    }
  }

  @Override
  @JsonProperty
  public ClusterBy clusterBy()
  {
    return clusterBy;
  }

  @Override
  public int partitionCount()
  {
    if (maxPartitions == 1) {
      return 1;
    } else {
      throw new ISE("Number of partitions not known for [%s] with maxPartitions[%d].", kind(), maxPartitions);
    }
  }

  @JsonProperty("partitions")
  public int getMaxPartitions()
  {
    return maxPartitions;
  }

  @Override
  @JsonInclude(value = JsonInclude.Include.CUSTOM, valueFilter = LimitHintJsonIncludeFilter.class)
  @JsonProperty
  public long limitHint()
  {
    return limitHint;
  }

  @Override

View on GitHub (pinned to 9b90983fd2)

Solutions

1. If a single partition is intended, configure maxPartitions = 1 so partitionCount() succeeds
2. Use getMaxPartitions() (the @JsonProperty accessor) instead of partitionCount() to read the configured cap
3. Treat partitionCount() as unavailable for this spec and handle it via the worker-stage logic that computes partitions dynamically

Example fix

// before
int count = shuffleSpec.partitionCount();
// after
int count = shuffleSpec.getMaxPartitions(); // or handle ISE when maxPartitions != 1

if (shuffleSpec instanceof GlobalSortMaxCountShuffleSpec
    && ((GlobalSortMaxCountShuffleSpec) shuffleSpec).getMaxPartitions() != 1) {
  // partitionCount() will throw; use getMaxPartitions() instead
}

static boolean partitionCountKnown(ShuffleSpec spec) {
  if (spec instanceof GlobalSortMaxCountShuffleSpec) {
    return ((GlobalSortMaxCountShuffleSpec) spec).getMaxPartitions() == 1;
  }
  return spec.canComputePartitionCount();
}

try {
  return spec.partitionCount();
} catch (IllegalStateException e) {
  return OptionalInt.empty(); // count determined at execution time
}

When it happens

Trigger: Calling partitionCount() on a GlobalSortMaxCountShuffleSpec whose maxPartitions is greater than 1 (e.g. code that assumes partitioning is known up front, such as broadcast-side sizing or partition-count reporting).

Common situations: Framework code querying a shuffle spec's partition count before the cluster-by statistics collector has run; distributed queries where maxPartitions was configured > 1.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Partition count must be at least 1
• Partition count must be 1 when adjustable is true, but was [
• ClusterBy key must be sortable
• Cannot bucket with %s partitioning
• ClusterBy key must be sortable

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/d626f1c46b278fb7. Report an issue: GitHub.