apache/iceberg · error · IllegalArgumentException

Cannot mix identity sort columns and a Zorder or Hilbert…

Error message

Cannot mix identity sort columns and a Zorder or Hilbert sort expression:

What it means

parseSortOrder also rejects combining plain identity sort columns with a Zorder or Hilbert expression in the same sort_order option. A space-filling curve ordering and an ordinary ascending/descending column ordering cannot be composed into one SortOrder, so the procedure fails fast rather than silently dropping terms.

Solutions

Use only identity columns, or only a single zorder()/hilbert() expression, in sort_order.
If you need extra ordering inside the curve, include those columns as terms within the zorder()/hilbert(...) call.
Run two rewrites (or use a regular sort_order) if plain-column ordering must be applied separately from curve ordering.

Example fix

// before
.option("sort_order", "ts, zorder(x, y)")
// after
.option("sort_order", "zorder(x, y, ts)")

Defensive patterns

Strategy: validation

Validate before calling

boolean hasCurve = sortOrder.contains("zorder(") || sortOrder.contains("hilbert(");
boolean hasPlain = sortOrder.split(",").length > 1 && !hasCurve /* plus parse check for bare columns outside curve parens */;

Try / catch

try {
    action.option("sort_order", sortOrder).execute();
} catch (IllegalArgumentException e) {
    if (e.getMessage().startsWith("Cannot mix identity sort columns")) { /* move columns into the curve expression */ }
}

Prevention

When a curve expression is present, list ALL columns inside zorder(...)/hilbert(...)
Never append extra 'ORDER BY'-style columns after a curve expression
Add a config-lint step in CI for rewrite job options

When it happens

Trigger: Calling rewrite_data_files with sort_order like 'id, zorder(x, y)' or 'hilbert(a), ts DESC' — any mix of bare column names (with optional ASC/DESC) and a zorder()/hilbert() expression in one sort_order string, when strategy='sort'.

Common situations: Adding a timestamp column 'for recency' onto an existing zorder config; assuming zorder accepts trailing identity columns like a SQL ORDER BY; migrating configs from engines where clustering specs can mix transforms and plain columns.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b2e480c55b426536. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java:210

    ExtendedParser.parseSortOrder(spark(), sortOrderString)
        .forEach(
            field -> {
              if (field.term() instanceof Zorder) {
                zOrderTerms.add((Zorder) field.term());
              } else if (field.term() instanceof Hilbert) {
                hilbertTerms.add((Hilbert) field.term());
              } else {
                sortOrderFields.add(field);
              }
            });

    if (!zOrderTerms.isEmpty() && !hilbertTerms.isEmpty()) {
      throw new IllegalArgumentException(
          "Cannot mix Zorder and Hilbert sort expressions: " + sortOrderString);
    }

    if ((!zOrderTerms.isEmpty() || !hilbertTerms.isEmpty()) && !sortOrderFields.isEmpty()) {
      throw new IllegalArgumentException(
          "Cannot mix identity sort columns and a Zorder or Hilbert sort expression: "
              + sortOrderString);
    }
  }

  private RewriteDataFilesSparkAction applySortStrategy(
      RewriteDataFilesSparkAction action,
      List<Zorder> zOrderTerms,
      List<Hilbert> hilbertTerms,
      List<ExtendedParser.RawOrderField> sortOrderFields,
      Schema schema) {
    if (!zOrderTerms.isEmpty()) {
      String[] columnNames =
          zOrderTerms.stream()
              .flatMap(zOrder -> zOrder.refs().stream().map(NamedReference::name))
              .toArray(String[]::new);
      return action.zOrder(columnNames);
    } else if (!hilbertTerms.isEmpty()) {

View on GitHub (pinned to 86d9c8fc54)