apache/druid · error · IllegalArgumentException

Cannot bucket with %s partitioning (clusterBy = %s)

Error message

Cannot bucket with %s partitioning (clusterBy = %s)

What it means

Bucket-by keys (clusterBy.getBucketByCount() > 0) are only supported by GlobalSortTargetSizeShuffleSpec. When a HashShuffleSpec is constructed with a ClusterBy containing bucket-by keys, the constructor rejects it with IAE, including the clusterBy for diagnosis.

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/HashShuffleSpec.java:55

  @JsonCreator
  public HashShuffleSpec(
      @JsonProperty("clusterBy") final ClusterBy clusterBy,
      @JsonProperty("partitions") final int numPartitions,
      @JsonProperty("adjustable") final boolean adjustable
  )
  {
    this.clusterBy = clusterBy;
    this.numPartitions = numPartitions;
    this.adjustable = adjustable;

    if (adjustable && numPartitions != 1) {
      throw new IAE("Partition count must be 1 when adjustable is true, but was [%d]", numPartitions);
    }

    if (clusterBy.getBucketByCount() > 0) {
      // Only GlobalSortTargetSizeShuffleSpec supports bucket-by.
      throw new IAE("Cannot bucket with %s partitioning (clusterBy = %s)", TYPE, clusterBy);
    }
  }

  @Override
  public ShuffleKind kind()
  {
    return clusterBy.sortable() && !clusterBy.isEmpty() ? ShuffleKind.HASH_LOCAL_SORT : ShuffleKind.HASH;
  }

  @Override
  @JsonProperty
  public ClusterBy clusterBy()
  {
    return clusterBy;
  }

  @Override
  @JsonProperty("partitions")

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Remove the bucket-by keys from the ClusterBy before creating the hash shuffle spec
2. Use GlobalSortTargetSizeShuffleSpec (TARGET_SIZE kind) if bucket-by is required
3. Choose a sort-based partitioning strategy rather than hash when bucket keys are present

Example fix

// before
new HashShuffleSpec(clusterByWithBucketBy, numPartitions, false);
// after
new HashShuffleSpec(clusterBy.withoutBucketBy(), numPartitions, false);

if (clusterBy.getBucketByCount() > 0) {
  throw new IllegalArgumentException("Bucket-by requires GlobalSortTargetSizeShuffleSpec, not hash");
}
new HashShuffleSpec(clusterBy, numPartitions, adjustable);

static boolean hashSupportsClusterBy(ClusterBy clusterBy) {
  return clusterBy == null || clusterBy.getBucketByCount() == 0;
}

try {
  spec = new HashShuffleSpec(clusterBy, numPartitions, adjustable);
} catch (IllegalArgumentException e) {
  spec = new GlobalSortTargetSizeShuffleSpec(clusterBy, targetSize, false);
}

When it happens

Trigger: Building a HashShuffleSpec (HASH shuffle kind via ShuffleSpecFactory.create) from a ClusterBy with bucket-by columns, e.g. an MSQ query using time-bucketed partitioning with hash shuffling.

Common situations: Queries that specify bucket (time) partitioning while the engine selects hash partitioning; configuration mixing CLUSTERED BY time columns with hash-based distribution.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Cannot bucket with %s partitioning
• ClusterBy key must be sortable
• Partition count must be 1 when adjustable is true, but was [
• Partition count must be at least 1
• Partition count must be 1 when adjustable is true, but was [

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/23f20463d3d8d480. Report an issue: GitHub.