{"record":{"id":"40b615d46891eeee","repo":"apache/druid","slug":"number-of-partitions-not-known-for-s","errorCode":null,"errorMessage":"Number of partitions not known for [%s].","messagePattern":"Number of partitions not known for \\[(.+?)\\]\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"multi-stage-query/src/main/java/org/apache/druid/msq/kernel/GlobalSortTargetSizeShuffleSpec.java","lineNumber":88,"sourceCode":"\n  @Override\n  @JsonProperty(\"aggregate\")\n  @JsonInclude(JsonInclude.Include.NON_DEFAULT)\n  public boolean doesAggregate()\n  {\n    return aggregate;\n  }\n\n  @Override\n  public boolean mustGatherResultKeyStatistics()\n  {\n    return true;\n  }\n\n  @Override\n  public int partitionCount()\n  {\n    throw new ISE(\"Number of partitions not known for [%s].\", kind());\n  }\n\n  @Override\n  public Either<Long, ClusterByPartitions> generatePartitionsForGlobalSort(\n      @Nullable final ClusterByStatisticsCollector collector,\n      final int maxNumPartitions\n  )\n  {\n    final long expectedPartitions = collector.estimatedTotalWeight() / targetSize;\n\n    if (expectedPartitions > maxNumPartitions) {\n      return Either.error(expectedPartitions);\n    } else {\n      collector.logSketches();\n      final ClusterByPartitions generatedPartitions = collector.generatePartitionsWithTargetWeight(targetSize);\n      if (generatedPartitions.size() <= maxNumPartitions) {\n        return Either.value(generatedPartitions);\n      } else {","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/multi-stage-query/src/main/java/org/apache/druid/msq/kernel/GlobalSortTargetSizeShuffleSpec.java#L70-L106","documentation":"For target-size shuffles, the exact number of partitions cannot be known until the global sort has processed the cluster-by statistics; partitions are generated dynamically via generatePartitionsForGlobalSort. partitionCount() therefore always throws ISE for this spec — it is intentionally unsupported.","triggerScenarios":"Any call to partitionCount() on a GlobalSortTargetSizeShuffleSpec instance, regardless of configuration (e.g. framework code that reports partition counts before execution).","commonSituations":"Code that uniformly asks shuffle specs for a static partition count; mixing specs where some (MAX_COUNT with maxPartitions=1) can answer and target-size cannot.","solutions":["Do not call partitionCount() on target-size specs; use generatePartitionsForGlobalSort(collector, maxNumPartitions) after collecting statistics","Check the shuffle kind with instanceof/kind() before calling partitionCount()","Use the targetSize configuration to reason about expected partition count rather than partitionCount()"],"exampleFix":"// before\nint partitions = shuffleSpec.partitionCount();\n// after\nif (shuffleSpec instanceof GlobalSortTargetSizeShuffleSpec) {\n  Either<Long, ClusterByPartitions> p = shuffleSpec.generatePartitionsForGlobalSort(collector, maxNumPartitions);\n} else {\n  int partitions = shuffleSpec.partitionCount();\n}","handlingStrategy":"type-guard","validationCode":"if (shuffleSpec instanceof GlobalSortTargetSizeShuffleSpec) {\n  // partitionCount() is always unsupported; use generatePartitionsForGlobalSort\n}","typeGuard":"static boolean hasStaticPartitionCount(ShuffleSpec spec) {\n  return !(spec instanceof GlobalSortTargetSizeShuffleSpec);\n}","tryCatchPattern":"try {\n  return spec.partitionCount();\n} catch (IllegalStateException e) {\n  return computeFromTargetSize(spec); // derive dynamically\n}","preventionTips":["Never call partitionCount() on target-size specs","Route through generatePartitionsForGlobalSort after collecting ClusterByStatisticsCollector results","Dispatch on spec kind before querying partition counts"],"tags":["msq","shuffle-spec","unsupported-operation","partition-count"],"backgroundTag":"unsupported-operation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}