apache/iceberg · error · IllegalArgumentException

Cannot mix identity sort columns and a Zorder or Hilbert sor

Error message

Cannot mix identity sort columns and a Zorder or Hilbert sort expression: ${sortOrderString}

What it means

parseSortOrder in RewriteDataFilesProcedure also rejects combining plain (identity) sort columns with a zorder(...) or hilbert(...) expression, since the two are distinct ordering modes implemented differently. This IllegalArgumentException includes the full sort_order string.

Source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java:210

    ExtendedParser.parseSortOrder(spark(), sortOrderString)
        .forEach(
            field -> {
              if (field.term() instanceof Zorder) {
                zOrderTerms.add((Zorder) field.term());
              } else if (field.term() instanceof Hilbert) {
                hilbertTerms.add((Hilbert) field.term());
              } else {
                sortOrderFields.add(field);
              }
            });

    if (!zOrderTerms.isEmpty() && !hilbertTerms.isEmpty()) {
      throw new IllegalArgumentException(
          "Cannot mix Zorder and Hilbert sort expressions: " + sortOrderString);
    }

    if ((!zOrderTerms.isEmpty() || !hilbertTerms.isEmpty()) && !sortOrderFields.isEmpty()) {
      throw new IllegalArgumentException(
          "Cannot mix identity sort columns and a Zorder or Hilbert sort expression: "
              + sortOrderString);
    }
  }

  private RewriteDataFilesSparkAction applySortStrategy(
      RewriteDataFilesSparkAction action,
      List<Zorder> zOrderTerms,
      List<Hilbert> hilbertTerms,
      List<ExtendedParser.RawOrderField> sortOrderFields,
      Schema schema) {
    if (!zOrderTerms.isEmpty()) {
      String[] columnNames =
          zOrderTerms.stream()
              .flatMap(zOrder -> zOrder.refs().stream().map(NamedReference::name))
              .toArray(String[]::new);
      return action.zOrder(columnNames);
    } else if (!hilbertTerms.isEmpty()) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

Remove plain column terms and keep only zorder(...)/hilbert(...) terms, or vice versa.
Use sort_order => 'zorder(b, c, d)' folding the extra column into the spatial expression if ordering on all columns is needed.
Run two separate rewrite passes: one identity-sorted, one spatially sorted.

Example fix

// before
call rewrite_data_files(table => 'db.t', sort_order => 'a, zorder(b, c)')
// after
call rewrite_data_files(table => 'db.t', sort_order => 'zorder(a, b, c)')

Defensive patterns

Strategy: validation

Validate before calling

val spatial = sortOrder.contains("zorder(") || sortOrder.contains("hilbert(")
val plainCols = sortOrder.split(",").map(_.trim).filterNot(_.matches("(zorder|hilbert)\\(.*\\)"))
require(!(spatial && plainCols.exists(_.nonEmpty)), "no identity cols with zorder/hilbert")

Prevention

Fold extra columns into the zorder/hilbert expression
Keep separate configs for identity-sorted and spatially-sorted tables
Validate sort_order strings before passing to CALL

When it happens

Trigger: Calling rewrite_data_files with sort_order mixing plain column names and spatial functions, e.g. sort_order => 'a, zorder(b, c)'.

Common situations: Appending a new column to an existing zorder sort_order string; copying a sort spec from an identity-sort config into a zorder one.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/52919a2f67707d2f. Report an issue: GitHub.