apache/iceberg · error · IllegalArgumentException

Cannot mix identity sort columns and a Zorder sort expressio

Error message

Cannot mix identity sort columns and a Zorder sort expression: ${sortOrderString}

What it means

The RewriteDataFiles procedure throws this when a user combines identity sort columns with a Zorder sort expression in one rewrite operation. The underlying SparkAction does not yet support mixing these two sort strategies, so the procedure rejects the combination up front rather than producing incorrect results. The message includes the requested sort order string for diagnosis.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/RewriteDataFilesProcedure.java:172

  private RewriteDataFiles checkAndApplyStrategy(
      RewriteDataFiles action, String strategy, String sortOrderString, Schema schema) {
    List<Zorder> zOrderTerms = Lists.newArrayList();
    List<ExtendedParser.RawOrderField> sortOrderFields = Lists.newArrayList();
    if (sortOrderString != null) {
      ExtendedParser.parseSortOrder(spark(), sortOrderString)
          .forEach(
              field -> {
                if (field.term() instanceof Zorder) {
                  zOrderTerms.add((Zorder) field.term());
                } else {
                  sortOrderFields.add(field);
                }
              });

      if (!zOrderTerms.isEmpty() && !sortOrderFields.isEmpty()) {
        // TODO: we need to allow this in future when SparkAction has handling for this.
        throw new IllegalArgumentException(
            "Cannot mix identity sort columns and a Zorder sort expression: " + sortOrderString);
      }
    }

    // caller of this function ensures that between strategy and sortOrder, at least one of them is
    // not null.
    if (strategy == null || strategy.equalsIgnoreCase("sort")) {
      if (!zOrderTerms.isEmpty()) {
        String[] columnNames =
            zOrderTerms.stream()
                .flatMap(zOrder -> zOrder.refs().stream().map(NamedReference::name))
                .toArray(String[]::new);
        return action.zOrder(columnNames);
      } else if (!sortOrderFields.isEmpty()) {
        return action.sort(buildSortOrder(sortOrderFields, schema));
      } else {
        return action.sort();
      }

View on GitHub (pinned to 86d9c8fc54)

Solutions

Use either only identity sort columns or only Zorder terms in sort_order, not both.
If Zorder is wanted for most columns, include all sort columns in a single zorder(...) expression.
Wait for upstream support tracked by the TODO in RewriteDataFilesProcedure, or split the rewrite into two passes (Zorder pass then sort pass) via separate procedure calls.
Use strategy => 'binpack' with no sort_order if compaction without sorting is acceptable.

Example fix

// before
CALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', sort_order => 'zorder(a,b),c');
// after
CALL iceberg.system.rewrite_data_files(table => 'db.t', strategy => 'sort', sort_order => 'zorder(a,b,c)');

Defensive patterns

Strategy: validation

Validate before calling

String sortOrder = options.get("sort_order");
boolean hasZorder = sortOrder != null && sortOrder.contains("zorder(");
boolean hasIdentity = sortOrder != null && sortOrder.replaceAll("zorder\\([^)]*\\)", "").matches(".*[a-zA-Z_].*");
if (hasZorder && hasIdentity) throw new IllegalArgumentException("Use only zorder or only identity sort columns");

Prevention

Keep sort_order either all zorder(...) or all plain columns.
Document the restriction in team runbooks for rewrite_data_files.
Watch Iceberg releases for the TODO that lifts this restriction.

When it happens

Trigger: Calling CALL iceberg.system.rewrite_data_files with strategy 'sort' where the sort_order option mixes Zorder (zorder(colA,colB)) and plain identity columns (colC), e.g. sort_order => 'zorder(a,b),c'.

Common situations: Users migrating from all-identity sort orders add a Zorder term incrementally, or copy sort order strings from examples that mix both, unaware the SparkAction lacks handling for the mix.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/4aa009165f978a8b. Report an issue: GitHub.