{"record":{"id":"b2e480c55b426536","repo":"apache/iceberg","slug":"cannot-mix-identity-sort-columns-and-a-zorder-or-h","errorCode":null,"errorMessage":"Cannot mix identity sort columns and a Zorder or Hilbert sort expression: ","messagePattern":"Cannot mix identity sort columns and a Zorder or Hilbert sort expression: ","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java","lineNumber":210,"sourceCode":"    ExtendedParser.parseSortOrder(spark(), sortOrderString)\n        .forEach(\n            field -> {\n              if (field.term() instanceof Zorder) {\n                zOrderTerms.add((Zorder) field.term());\n              } else if (field.term() instanceof Hilbert) {\n                hilbertTerms.add((Hilbert) field.term());\n              } else {\n                sortOrderFields.add(field);\n              }\n            });\n\n    if (!zOrderTerms.isEmpty() && !hilbertTerms.isEmpty()) {\n      throw new IllegalArgumentException(\n          \"Cannot mix Zorder and Hilbert sort expressions: \" + sortOrderString);\n    }\n\n    if ((!zOrderTerms.isEmpty() || !hilbertTerms.isEmpty()) && !sortOrderFields.isEmpty()) {\n      throw new IllegalArgumentException(\n          \"Cannot mix identity sort columns and a Zorder or Hilbert sort expression: \"\n              + sortOrderString);\n    }\n  }\n\n  private RewriteDataFilesSparkAction applySortStrategy(\n      RewriteDataFilesSparkAction action,\n      List<Zorder> zOrderTerms,\n      List<Hilbert> hilbertTerms,\n      List<ExtendedParser.RawOrderField> sortOrderFields,\n      Schema schema) {\n    if (!zOrderTerms.isEmpty()) {\n      String[] columnNames =\n          zOrderTerms.stream()\n              .flatMap(zOrder -> zOrder.refs().stream().map(NamedReference::name))\n              .toArray(String[]::new);\n      return action.zOrder(columnNames);\n    } else if (!hilbertTerms.isEmpty()) {","sourceCodeStart":192,"sourceCodeEnd":228,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java#L192-L228","documentation":"parseSortOrder also rejects combining plain identity sort columns with a Zorder or Hilbert expression in the same sort_order option. A space-filling curve ordering and an ordinary ascending/descending column ordering cannot be composed into one SortOrder, so the procedure fails fast rather than silently dropping terms.","triggerScenarios":"Calling rewrite_data_files with sort_order like 'id, zorder(x, y)' or 'hilbert(a), ts DESC' — any mix of bare column names (with optional ASC/DESC) and a zorder()/hilbert() expression in one sort_order string, when strategy='sort'.","commonSituations":"Adding a timestamp column 'for recency' onto an existing zorder config; assuming zorder accepts trailing identity columns like a SQL ORDER BY; migrating configs from engines where clustering specs can mix transforms and plain columns.","solutions":["Use only identity columns, or only a single zorder()/hilbert() expression, in sort_order.","If you need extra ordering inside the curve, include those columns as terms within the zorder()/hilbert(...) call.","Run two rewrites (or use a regular sort_order) if plain-column ordering must be applied separately from curve ordering."],"exampleFix":"// before\n.option(\"sort_order\", \"ts, zorder(x, y)\")\n// after\n.option(\"sort_order\", \"zorder(x, y, ts)\")","handlingStrategy":"validation","validationCode":"boolean hasCurve = sortOrder.contains(\"zorder(\") || sortOrder.contains(\"hilbert(\");\nboolean hasPlain = sortOrder.split(\",\").length > 1 && !hasCurve /* plus parse check for bare columns outside curve parens */;","typeGuard":null,"tryCatchPattern":"try {\n    action.option(\"sort_order\", sortOrder).execute();\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().startsWith(\"Cannot mix identity sort columns\")) { /* move columns into the curve expression */ }\n}","preventionTips":["When a curve expression is present, list ALL columns inside zorder(...)/hilbert(...)","Never append extra 'ORDER BY'-style columns after a curve expression","Add a config-lint step in CI for rewrite job options"],"tags":["spark","sort-order","config-validation"],"backgroundTag":"mutually-exclusive-options","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}