{"record":{"id":"4aa009165f978a8b","repo":"apache/iceberg","slug":"cannot-mix-identity-sort-columns-and-a-zorder-sort","errorCode":null,"errorMessage":"Cannot mix identity sort columns and a Zorder sort expression: ${sortOrderString}","messagePattern":"Cannot mix identity sort columns and a Zorder sort expression: (.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java","lineNumber":172,"sourceCode":"\n  private RewriteDataFiles checkAndApplyStrategy(\n      RewriteDataFiles action, String strategy, String sortOrderString, Schema schema) {\n    List<Zorder> zOrderTerms = Lists.newArrayList();\n    List<ExtendedParser.RawOrderField> sortOrderFields = Lists.newArrayList();\n    if (sortOrderString != null) {\n      ExtendedParser.parseSortOrder(spark(), sortOrderString)\n          .forEach(\n              field -> {\n                if (field.term() instanceof Zorder) {\n                  zOrderTerms.add((Zorder) field.term());\n                } else {\n                  sortOrderFields.add(field);\n                }\n              });\n\n      if (!zOrderTerms.isEmpty() && !sortOrderFields.isEmpty()) {\n        // TODO: we need to allow this in future when SparkAction has handling for this.\n        throw new IllegalArgumentException(\n            \"Cannot mix identity sort columns and a Zorder sort expression: \" + sortOrderString);\n      }\n    }\n\n    // caller of this function ensures that between strategy and sortOrder, at least one of them is\n    // not null.\n    if (strategy == null || strategy.equalsIgnoreCase(\"sort\")) {\n      if (!zOrderTerms.isEmpty()) {\n        String[] columnNames =\n            zOrderTerms.stream()\n                .flatMap(zOrder -> zOrder.refs().stream().map(NamedReference::name))\n                .toArray(String[]::new);\n        return action.zOrder(columnNames);\n      } else if (!sortOrderFields.isEmpty()) {\n        return action.sort(buildSortOrder(sortOrderFields, schema));\n      } else {\n        return action.sort();\n      }","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java#L154-L190","documentation":"The RewriteDataFiles procedure throws this when a user combines identity sort columns with a Zorder sort expression in one rewrite operation. The underlying SparkAction does not yet support mixing these two sort strategies, so the procedure rejects the combination up front rather than producing incorrect results. The message includes the requested sort order string for diagnosis.","triggerScenarios":"Calling CALL iceberg.system.rewrite_data_files with strategy 'sort' where the sort_order option mixes Zorder (zorder(colA,colB)) and plain identity columns (colC), e.g. sort_order => 'zorder(a,b),c'.","commonSituations":"Users migrating from all-identity sort orders add a Zorder term incrementally, or copy sort order strings from examples that mix both, unaware the SparkAction lacks handling for the mix.","solutions":["Use either only identity sort columns or only Zorder terms in sort_order, not both.","If Zorder is wanted for most columns, include all sort columns in a single zorder(...) expression.","Wait for upstream support tracked by the TODO in RewriteDataFilesProcedure, or split the rewrite into two passes (Zorder pass then sort pass) via separate procedure calls.","Use strategy => 'binpack' with no sort_order if compaction without sorting is acceptable."],"exampleFix":"// before\nCALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', sort_order => 'zorder(a,b),c');\n// after\nCALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', sort_order => 'zorder(a,b,c)');","handlingStrategy":"validation","validationCode":"String sortOrder = options.get(\"sort_order\");\nboolean hasZorder = sortOrder != null && sortOrder.contains(\"zorder(\");\nboolean hasIdentity = sortOrder != null && sortOrder.replaceAll(\"zorder\\\\([^)]*\\\\)\", \"\").matches(\".*[a-zA-Z_].*\");\nif (hasZorder && hasIdentity) throw new IllegalArgumentException(\"Use only zorder or only identity sort columns\");","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep sort_order either all zorder(...) or all plain columns.","Document the restriction in team runbooks for rewrite_data_files.","Watch Iceberg releases for the TODO that lifts this restriction."],"tags":["spark","rewrite-data-files","sort-order","zorder"],"backgroundTag":"mutually-exclusive-options","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}