apache/iceberg · error · IllegalArgumentException

Cannot mix identity sort columns and a Zorder sort expressio

Error message

Cannot mix identity sort columns and a Zorder sort expression: + sortOrderString

What it means

RewriteDataFilesProcedure supports either identity sort columns or a Z-order sort expression, but not both in one rewrite, because the underlying Spark RewriteDataFiles action has no combined handling for the two sort strategies. When both z_order_terms (Zorder by) and plain sort columns resolve to non-empty sets, checkAndApplyStrategy throws IllegalArgumentException.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java:175

  private RewriteDataFilesSparkAction checkAndApplyStrategy(
      RewriteDataFilesSparkAction action, String strategy, String sortOrderString, Schema schema) {
    List<Zorder> zOrderTerms = Lists.newArrayList();
    List<ExtendedParser.RawOrderField> sortOrderFields = Lists.newArrayList();
    if (sortOrderString != null) {
      ExtendedParser.parseSortOrder(spark(), sortOrderString)
          .forEach(
              field -> {
                if (field.term() instanceof Zorder) {
                  zOrderTerms.add((Zorder) field.term());
                } else {
                  sortOrderFields.add(field);
                }
              });

      if (!zOrderTerms.isEmpty() && !sortOrderFields.isEmpty()) {
        // TODO: we need to allow this in future when SparkAction has handling for this.
        throw new IllegalArgumentException(
            "Cannot mix identity sort columns and a Zorder sort expression: " + sortOrderString);
      }
    }

    // caller of this function ensures that between strategy and sortOrder, at least one of them is
    // not null.
    if (strategy == null || strategy.equalsIgnoreCase("sort")) {
      if (!zOrderTerms.isEmpty()) {
        String[] columnNames =
            zOrderTerms.stream()
                .flatMap(zOrder -> zOrder.refs().stream().map(NamedReference::name))
                .toArray(String[]::new);
        return action.zOrder(columnNames);
      } else if (!sortOrderFields.isEmpty()) {
        return action.sort(buildSortOrder(sortOrderFields, schema));
      } else {
        return action.sort();
      }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use only one: either z_order terms or sort_order columns, not both.
2. If a global-ish order is needed, pick Zorder alone, or express everything as sort_order columns.
3. Nested-sort within partitions: keep identity sort_order and drop the Zorder terms.
4. Track future Iceberg versions — the TODO notes this restriction may be lifted.

Example fix

-- before (both specified)
CALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', sort_order => 'id ASC', z_order => 'ts');
-- after (single strategy)
CALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', z_order => 'ts');

boolean hasZorder = zOrderTerms != null && !zOrderTerms.isEmpty(); boolean hasSort = sortOrderFields != null && !sortOrderFields.isEmpty(); if (hasZorder && hasSort) throw new IllegalArgumentException("choose either z_order or sort_order, not both");

try { rewriteDataFiles(...); } catch (IllegalArgumentException e) { if (e.getMessage().contains("Cannot mix identity sort")) { /* drop one of the options */ } else throw e; }

When it happens

Trigger: CALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', where/kind of options mixing z_order => 'a,b' with sort_order => 'c ASC, d DESC') so that both the Zorder term list and identity sort field list are populated.

Common situations: Combining leftover sort_order options from a previous invocation with newly added z_order terms; thinking Zorder and sort compose; copy-pasted procedure templates that set both.

Related errors

• Cannot mix Zorder and Hilbert sort expressions: ${sortOrderS
• Cannot mix identity sort columns and a Zorder or Hilbert sor
• Unable to parse sortOrder: %s
• Transform is not supported: ${transform}
• Cannot specify the 'sort-order' because it's a reserved tabl

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5366565597ab0dca. Report an issue: GitHub.