{"record":{"id":"5366565597ab0dca","repo":"apache/iceberg","slug":"cannot-mix-identity-sort-columns-and-a-zorder-sort-536656","errorCode":null,"errorMessage":"Cannot mix identity sort columns and a Zorder sort expression: + sortOrderString","messagePattern":"Cannot mix identity sort columns and a Zorder sort expression: \\+ sortOrderString","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java","lineNumber":175,"sourceCode":"\n  private RewriteDataFilesSparkAction checkAndApplyStrategy(\n      RewriteDataFilesSparkAction action, String strategy, String sortOrderString, Schema schema) {\n    List<Zorder> zOrderTerms = Lists.newArrayList();\n    List<ExtendedParser.RawOrderField> sortOrderFields = Lists.newArrayList();\n    if (sortOrderString != null) {\n      ExtendedParser.parseSortOrder(spark(), sortOrderString)\n          .forEach(\n              field -> {\n                if (field.term() instanceof Zorder) {\n                  zOrderTerms.add((Zorder) field.term());\n                } else {\n                  sortOrderFields.add(field);\n                }\n              });\n\n      if (!zOrderTerms.isEmpty() && !sortOrderFields.isEmpty()) {\n        // TODO: we need to allow this in future when SparkAction has handling for this.\n        throw new IllegalArgumentException(\n            \"Cannot mix identity sort columns and a Zorder sort expression: \" + sortOrderString);\n      }\n    }\n\n    // caller of this function ensures that between strategy and sortOrder, at least one of them is\n    // not null.\n    if (strategy == null || strategy.equalsIgnoreCase(\"sort\")) {\n      if (!zOrderTerms.isEmpty()) {\n        String[] columnNames =\n            zOrderTerms.stream()\n                .flatMap(zOrder -> zOrder.refs().stream().map(NamedReference::name))\n                .toArray(String[]::new);\n        return action.zOrder(columnNames);\n      } else if (!sortOrderFields.isEmpty()) {\n        return action.sort(buildSortOrder(sortOrderFields, schema));\n      } else {\n        return action.sort();\n      }","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java#L157-L193","documentation":"RewriteDataFilesProcedure supports either identity sort columns or a Z-order sort expression, but not both in one rewrite, because the underlying Spark RewriteDataFiles action has no combined handling for the two sort strategies. When both z_order_terms (Zorder by) and plain sort columns resolve to non-empty sets, checkAndApplyStrategy throws IllegalArgumentException.","triggerScenarios":"CALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', where/kind of options mixing z_order => 'a,b' with sort_order => 'c ASC, d DESC') so that both the Zorder term list and identity sort field list are populated.","commonSituations":"Combining leftover sort_order options from a previous invocation with newly added z_order terms; thinking Zorder and sort compose; copy-pasted procedure templates that set both.","solutions":["Use only one: either z_order terms or sort_order columns, not both.","If a global-ish order is needed, pick Zorder alone, or express everything as sort_order columns.","Nested-sort within partitions: keep identity sort_order and drop the Zorder terms.","Track future Iceberg versions — the TODO notes this restriction may be lifted."],"exampleFix":"-- before (both specified)\nCALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', sort_order => 'id ASC', z_order => 'ts');\n-- after (single strategy)\nCALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', z_order => 'ts');","handlingStrategy":"validation","validationCode":"boolean hasZorder = zOrderTerms != null && !zOrderTerms.isEmpty(); boolean hasSort = sortOrderFields != null && !sortOrderFields.isEmpty(); if (hasZorder && hasSort) throw new IllegalArgumentException(\"choose either z_order or sort_order, not both\");","typeGuard":null,"tryCatchPattern":"try { rewriteDataFiles(...); } catch (IllegalArgumentException e) { if (e.getMessage().contains(\"Cannot mix identity sort\")) { /* drop one of the options */ } else throw e; }","preventionTips":["Never set z_order and sort_order options together.","Keep procedure call templates with only one sort mechanism.","Review generated SQL for leftover options from previous calls."],"tags":["spark","compaction","sort-order"],"backgroundTag":"mutually-exclusive-options","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}