{"record":{"id":"1a87bd897c6d9ed3","repo":"apache/druid","slug":"clusterby-key-must-be-sortable-1a87bd","errorCode":null,"errorMessage":"ClusterBy key must be sortable","messagePattern":"ClusterBy key must be sortable","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"multi-stage-query/src/main/java/org/apache/druid/msq/kernel/GlobalSortTargetSizeShuffleSpec.java","lineNumber":61,"sourceCode":"  public static final String TYPE = \"targetSize\";\n\n  private final ClusterBy clusterBy;\n  private final long targetSize;\n  private final boolean aggregate;\n\n  @JsonCreator\n  public GlobalSortTargetSizeShuffleSpec(\n      @JsonProperty(\"clusterBy\") final ClusterBy clusterBy,\n      @JsonProperty(\"targetSize\") final long targetSize,\n      @JsonProperty(\"aggregate\") final boolean aggregate\n  )\n  {\n    this.clusterBy = Preconditions.checkNotNull(clusterBy, \"clusterBy\");\n    this.targetSize = targetSize;\n    this.aggregate = aggregate;\n\n    if (!clusterBy.sortable()) {\n      throw new IAE(\"ClusterBy key must be sortable\");\n    }\n  }\n\n  @Override\n  public ShuffleKind kind()\n  {\n    return ShuffleKind.GLOBAL_SORT;\n  }\n\n  @Override\n  @JsonProperty(\"aggregate\")\n  @JsonInclude(JsonInclude.Include.NON_DEFAULT)\n  public boolean doesAggregate()\n  {\n    return aggregate;\n  }\n\n  @Override","sourceCodeStart":43,"sourceCodeEnd":79,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/multi-stage-query/src/main/java/org/apache/druid/msq/kernel/GlobalSortTargetSizeShuffleSpec.java#L43-L79","documentation":"GlobalSortTargetSizeShuffleSpec requires that all cluster-by keys be sortable so results can be globally sorted and split into target-size partitions. The constructor validates clusterBy.sortable() and throws IAE if any cluster-by column lacks a sortable key representation (e.g. COMPLEX/unsupported types).","triggerScenarios":"Constructing GlobalSortTargetSizeShuffleSpec (via ShuffleSpecFactory.create with kind=TARGET_SIZE) with a ClusterBy whose key columns include non-sortable types (e.g. nested data / complex serializer keys).","commonSituations":"MSQ queries that cluster by complex columns (e.g. arrays, sketches, nested data) while requesting target-size partitioning; schema changes making previously used cluster columns non-sortable.","solutions":["Remove the non-sortable column from the cluster-by keys or wrap/replace it with a sortable expression (e.g. cast or stringify the complex value)","Cluster by a primitive dimension (string/long/double) instead of a complex column","Use a different shuffle kind that does not require global sorting"],"exampleFix":"// before\nClusterBy clusterBy = new ClusterBy(ImmutableList.of(Expr.of(\"sketchColumn\")), 0, 0);\nnew GlobalSortTargetSizeShuffleSpec(clusterBy, targetSize, false);\n// after: cluster on a sortable expression\nClusterBy clusterBy = new ClusterBy(ImmutableList.of(Expr.of(\"JSON_VALUE(metrics, '$.id')\")), 0, 0);\nnew GlobalSortTargetSizeShuffleSpec(clusterBy, targetSize, false);","handlingStrategy":"validation","validationCode":"for (ClusterBy.ColumnAndBucket column : clusterBy.getColumns()) {\n  if (!SortableTypes.isSortable(column.columnType())) {\n    throw new IllegalArgumentException(\"Cluster-by column '\" + column + \"' is not sortable\");\n  }\n}","typeGuard":"static boolean clusterBySortable(ClusterBy clusterBy) {\n  return clusterBy != null && clusterBy.sortable();\n}","tryCatchPattern":"try {\n  spec = new GlobalSortTargetSizeShuffleSpec(clusterBy, targetSize, aggregate);\n} catch (IllegalArgumentException e) {\n  // replace non-sortable columns or fall back to hash shuffle\n  spec = new HashShuffleSpec(clusterBy.withoutBucketBy(), 1, true);\n}","preventionTips":["Cluster only on string/long/double/float columns, not complex/nested types","Call clusterBy.sortable() as a pre-check before building sort-based specs","Cast or wrap complex columns with a sortable expression before clustering"],"tags":["msq","shuffle-spec","sortable","invalid-argument"],"backgroundTag":"invalid-argument-value","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}