apache/druid · error · IllegalArgumentException

ClusterBy key must be sortable

Error message

ClusterBy key must be sortable

What it means

GlobalSortTargetSizeShuffleSpec requires that all cluster-by keys be sortable so results can be globally sorted and split into target-size partitions. The constructor validates clusterBy.sortable() and throws IAE if any cluster-by column lacks a sortable key representation (e.g. COMPLEX/unsupported types).

Source

Thrown at multi-stage-query/src/main/java/org/apache/druid/msq/kernel/GlobalSortTargetSizeShuffleSpec.java:61

  public static final String TYPE = "targetSize";

  private final ClusterBy clusterBy;
  private final long targetSize;
  private final boolean aggregate;

  @JsonCreator
  public GlobalSortTargetSizeShuffleSpec(
      @JsonProperty("clusterBy") final ClusterBy clusterBy,
      @JsonProperty("targetSize") final long targetSize,
      @JsonProperty("aggregate") final boolean aggregate
  )
  {
    this.clusterBy = Preconditions.checkNotNull(clusterBy, "clusterBy");
    this.targetSize = targetSize;
    this.aggregate = aggregate;

    if (!clusterBy.sortable()) {
      throw new IAE("ClusterBy key must be sortable");
    }
  }

  @Override
  public ShuffleKind kind()
  {
    return ShuffleKind.GLOBAL_SORT;
  }

  @Override
  @JsonProperty("aggregate")
  @JsonInclude(JsonInclude.Include.NON_DEFAULT)
  public boolean doesAggregate()
  {
    return aggregate;
  }

  @Override

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Remove the non-sortable column from the cluster-by keys or wrap/replace it with a sortable expression (e.g. cast or stringify the complex value)
2. Cluster by a primitive dimension (string/long/double) instead of a complex column
3. Use a different shuffle kind that does not require global sorting

Example fix

// before
ClusterBy clusterBy = new ClusterBy(ImmutableList.of(Expr.of("sketchColumn")), 0, 0);
new GlobalSortTargetSizeShuffleSpec(clusterBy, targetSize, false);
// after: cluster on a sortable expression
ClusterBy clusterBy = new ClusterBy(ImmutableList.of(Expr.of("JSON_VALUE(metrics, '$.id')")), 0, 0);
new GlobalSortTargetSizeShuffleSpec(clusterBy, targetSize, false);

for (ClusterBy.ColumnAndBucket column : clusterBy.getColumns()) {
  if (!SortableTypes.isSortable(column.columnType())) {
    throw new IllegalArgumentException("Cluster-by column '" + column + "' is not sortable");
  }
}

static boolean clusterBySortable(ClusterBy clusterBy) {
  return clusterBy != null && clusterBy.sortable();
}

try {
  spec = new GlobalSortTargetSizeShuffleSpec(clusterBy, targetSize, aggregate);
} catch (IllegalArgumentException e) {
  // replace non-sortable columns or fall back to hash shuffle
  spec = new HashShuffleSpec(clusterBy.withoutBucketBy(), 1, true);
}

When it happens

Trigger: Constructing GlobalSortTargetSizeShuffleSpec (via ShuffleSpecFactory.create with kind=TARGET_SIZE) with a ClusterBy whose key columns include non-sortable types (e.g. nested data / complex serializer keys).

Common situations: MSQ queries that cluster by complex columns (e.g. arrays, sketches, nested data) while requesting target-size partitioning; schema changes making previously used cluster columns non-sortable.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Cannot bucket with %s partitioning
• Partition count must be 1 when adjustable is true, but was [
• Cannot bucket with %s partitioning (clusterBy = %s)
• Partition count must be at least 1
• Partition count must be 1 when adjustable is true, but was [

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/1a87bd897c6d9ed3. Report an issue: GitHub.