apache/druid · error · IllegalArgumentException

Cannot bucket with %s partitioning

Error message

Cannot bucket with %s partitioning

What it means

GlobalSortMaxCountShuffleSpec only supports shuffle specs whose ClusterBy has no bucket-by keys; bucket-by partitioning is reserved exclusively for GlobalSortTargetSizeShuffleSpec. When a ClusterBy with bucketBy count > 0 is used to construct this shuffle spec (during ShuffleSpecFactory.create), the constructor throws IAE because it cannot represent bucketed partitioning.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/GlobalSortMaxCountShuffleSpec.java:78

    this.aggregate = aggregate;
    this.limitHint = limitHint == null ? UNLIMITED : limitHint;
    this.adjustable = adjustable;

    if (maxPartitions < 1) {
      throw new IAE("Partition count must be at least 1");
    }

    if (adjustable && maxPartitions != 1) {
      throw new IAE("Partition count must be 1 when adjustable is true, but was [%d]", maxPartitions);
    }

    if (!clusterBy.sortable()) {
      throw new IAE("ClusterBy key must be sortable");
    }

    if (clusterBy.getBucketByCount() > 0) {
      // Only GlobalSortTargetSizeShuffleSpec supports bucket-by.
      throw new IAE("Cannot bucket with %s partitioning", TYPE);
    }
  }

  public GlobalSortMaxCountShuffleSpec(
      final ClusterBy clusterBy,
      final int maxPartitions,
      final boolean aggregate,
      final Long limitHint
  )
  {
    this(clusterBy, maxPartitions, aggregate, limitHint, false);
  }

  @Override
  public ShuffleKind kind()
  {
    return ShuffleKind.GLOBAL_SORT;
  }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Remove bucket-by keys from the ClusterBy (drop the bucket columns in the query's clustering/partitioning spec)
2. Use GlobalSortTargetSizeShuffleSpec (TARGET_SIZE shuffle kind) if bucket-by partitioning is required
3. Adjust the query context/partitioning parameters so maxCount shuffle is not combined with bucket-by

Example fix

// before: bucket-by columns with MAX_COUNT shuffle
new GlobalSortMaxCountShuffleSpec(clusterByWithBucketBy, maxPartitions);
// after: either strip bucket keys
ClusterBy sortableClusterBy = clusterBy.withoutBucketBy();
new GlobalSortMaxCountShuffleSpec(sortableClusterBy, maxPartitions);
// or use TARGET_SIZE shuffle which supports bucket-by

if (clusterBy.getBucketByCount() > 0) {
  throw new IllegalArgumentException("Use GlobalSortTargetSizeShuffleSpec (TARGET_SIZE) for bucket-by partitioning");
}
new GlobalSortMaxCountShuffleSpec(clusterBy, maxPartitions);

static boolean supportsMaxCount(ClusterBy clusterBy) {
  return clusterBy != null && clusterBy.getBucketByCount() == 0;
}

try {
  spec = new GlobalSortMaxCountShuffleSpec(clusterBy, maxPartitions);
} catch (IllegalArgumentException e) {
  spec = new GlobalSortTargetSizeShuffleSpec(clusterBy, targetSize, false);
}

When it happens

Trigger: Building a shuffle spec via ShuffleSpecFactory.create with kind=MAX_COUNT and a ClusterBy whose getBucketByCount() > 0 (i.e. the query specifies bucket-by columns for a global-sort max-count shuffle).

Common situations: MSQ queries that specify a partitioning/DISTRIBUTED OVER bucket columns while using the max-count global sort shuffle; misconfigured query context choosing maxCount partitioning together with time-bucketed clustering.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Cannot bucket with %s partitioning (clusterBy = %s)
• ClusterBy key must be sortable
• Partition count must be 1 when adjustable is true, but was [
• Partition count must be at least 1
• Partition count must be 1 when adjustable is true, but was [

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/b212382b989452d1. Report an issue: GitHub.