apache/druid · error · IllegalStateException

Number of partitions not known for

Error message

Number of partitions not known for [%s] with maxPartitions[%d].

What it means

GlobalSortMaxCountShuffleSpec.partitionCount() can only return a definitive count when maxPartitions == 1. For any other maxPartitions the actual number of partitions is determined only after stage execution, so calling partitionCount() throws ISE. Callers must use generatePartitionsForGlobalSort or maxPartitions instead.

Solutions

1. If a single partition is intended, configure maxPartitions = 1 so partitionCount() succeeds
2. Use getMaxPartitions() (the @JsonProperty accessor) instead of partitionCount() to read the configured cap
3. Treat partitionCount() as unavailable for this spec and handle it via the worker-stage logic that computes partitions dynamically

Example fix

// before
int count = shuffleSpec.partitionCount();
// after
int count = shuffleSpec.getMaxPartitions(); // or handle ISE when maxPartitions != 1

if (shuffleSpec instanceof GlobalSortMaxCountShuffleSpec
    && ((GlobalSortMaxCountShuffleSpec) shuffleSpec).getMaxPartitions() != 1) {
  // partitionCount() will throw; use getMaxPartitions() instead
}

static boolean partitionCountKnown(ShuffleSpec spec) {
  if (spec instanceof GlobalSortMaxCountShuffleSpec) {
    return ((GlobalSortMaxCountShuffleSpec) spec).getMaxPartitions() == 1;
  }
  return spec.canComputePartitionCount();
}

try {
  return spec.partitionCount();
} catch (IllegalStateException e) {
  return OptionalInt.empty(); // count determined at execution time
}

When it happens

Trigger: Calling partitionCount() on a GlobalSortMaxCountShuffleSpec whose maxPartitions is greater than 1 (e.g. code that assumes partitioning is known up front, such as broadcast-side sizing or partition-count reporting).

Common situations: Framework code querying a shuffle spec's partition count before the cluster-by statistics collector has run; distributed queries where maxPartitions was configured > 1.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Cannot accept duplicate stage numbers
• Cannot bucket with partitioning
• Cannot bucket with partitioning (clusterBy = )
• ClusterBy key must be sortable
• ClusterBy key must be sortable

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/d626f1c46b278fb7. Report an issue: GitHub.

Appendix: source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/GlobalSortMaxCountShuffleSpec.java:146

        return Either.error((long) generatedPartitions.size());
      }
    }
  }

  @Override
  @JsonProperty
  public ClusterBy clusterBy()
  {
    return clusterBy;
  }

  @Override
  public int partitionCount()
  {
    if (maxPartitions == 1) {
      return 1;
    } else {
      throw new ISE("Number of partitions not known for [%s] with maxPartitions[%d].", kind(), maxPartitions);
    }
  }

  @JsonProperty("partitions")
  public int getMaxPartitions()
  {
    return maxPartitions;
  }

  @Override
  @JsonInclude(value = JsonInclude.Include.CUSTOM, valueFilter = LimitHintJsonIncludeFilter.class)
  @JsonProperty
  public long limitHint()
  {
    return limitHint;
  }

  @Override

View on GitHub (pinned to 9b90983fd2)