Sort order specified for job {} doesn't match any table sort

Error message

Sort order specified for job {} doesn't match any table sort orders, rewritten files will not be marked as sorted in the manifest files

What it means

SparkShufflingFileRewriteRunner sorts data according to the sort order given in the rewrite job spec. If that job sort order doesn't match any of the table's declared sort orders, SortOrderUtil.findTableSortOrder yields unsorted, so rewritten files cannot be flagged as sorted in their manifests; this warning explains that consequence.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/SparkShufflingFileRewriteRunner.java:132

            .format("iceberg")
            .option(SparkReadOptions.SCAN_TASK_SET_ID, groupId)
            .load(groupId);

    Dataset<Row> sortedDF =
        sortedDF(
            scanDF,
            sortFunction(
                fileGroup.fileScanTasks(),
                spec(fileGroup.outputSpecId()),
                fileGroup.expectedOutputFiles()));

    org.apache.iceberg.SortOrder sortOrderInJobSpec = sortOrder();

    org.apache.iceberg.SortOrder maybeMatchingTableSortOrder =
        SortOrderUtil.findTableSortOrder(table(), sortOrder());

    if (sortOrderInJobSpec.isSorted() && maybeMatchingTableSortOrder.isUnsorted()) {
      LOG.warn(
          "Sort order specified for job {} doesn't match any table sort orders, rewritten files will not be marked as sorted in the manifest files",
          Spark3Util.describe(sortOrderInJobSpec));
    }

    sortedDF
        .write()
        .format("iceberg")
        .option(SparkWriteOptions.REWRITTEN_FILE_SCAN_TASK_SET_ID, groupId)
        .option(SparkWriteOptions.TARGET_FILE_SIZE_BYTES, fileGroup.maxOutputFileSize())
        .option(SparkWriteOptions.USE_TABLE_DISTRIBUTION_AND_ORDERING, "false")
        .option(SparkWriteOptions.OUTPUT_SPEC_ID, fileGroup.outputSpecId())
        .option(SparkWriteOptions.OUTPUT_SORT_ORDER_ID, maybeMatchingTableSortOrder.orderId())
        .mode("append")
        .save(groupId);
  }

  private Function<Dataset<Row>, Dataset<Row>> sortFunction(
      List<FileScanTask> group, PartitionSpec outputSpec, int expectedOutputFiles) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

Align the job's sort spec with the table's declared sort order (same columns, transforms, directions, null ordering), or update the table's sort order to match.
Run ALTER TABLE ... WRITE ORDERED BY to set a table sort order that the rewrite can match.
Accept the warning if files don't need sort-order metadata (data is still physically sorted; only manifest annotation is skipped).
Verify the job's describe(sortOrder) output in the log against the table's current sort order via DESCRIBE TABLE / metadata.

Example fix

// before: job sort doesn't match table order, files unmarked as sorted
spark.sql("CALL cat.sys.rewrite_data_files(table => 'db.t', " +
  "strategy => 'sort', where => 'id > 0', " +
  "sort_order => 'id')");
// after: make table sort order match the job spec first
spark.sql("ALTER TABLE db.t WRITE ORDERED BY id");
spark.sql("CALL cat.sys.rewrite_data_files(table => 'db.t', " +
  "strategy => 'sort', sort_order => 'id')");

Defensive patterns

Strategy: validation

Validate before calling

SortOrder jobOrder = sortOrder();
SortOrder tableOrder = SortOrderUtil.findTableSortOrder(table(), jobOrder);
if (jobOrder.isSorted() && tableOrder.isUnsorted()) {
  // fix job spec or ALTER TABLE WRITE ORDERED BY before rewriting
}

Prevention

Declare the table's sort order (ALTER TABLE ... WRITE ORDERED BY) matching your rewrite job spec.
Include transforms, direction, and null-order in comparisons — any mismatch invalidates the match.
After changing table sort order, update rewrite job configs.
Check the log's describe(sortOrder) output against table metadata before large rewrites.

When it happens

Trigger: Running rewrite_data_files with a sort_strategy/sort order (e.g. custom sort columns via procedure options) whose org.apache.iceberg.SortOrder is sorted but does not equal any sortOrder declared in table metadata, so SortOrderUtil.findTableSortOrder returns unsorted.

Common situations: Sort columns/transform/null-order in the job spec differing from the table's REPLACE/ALTER ... SORT BY order; table sort order changed after the job spec was written; using z-order strategy (job order never matches a table sort order by design).

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/034af947354e6e46. Report an issue: GitHub.