Sort order specified for job {} doesn't match any table sort

Error message

Sort order specified for job {} doesn't match any table sort orders, rewritten files will not be marked as sorted in the manifest files

What it means

WARN from SparkShufflingFileRewriteRunner.doRewrite: the sort order requested for the rewrite job does not match any of the table's declared sort orders, so while the data IS physically sorted by the job spec, the rewritten manifest entries cannot be marked with that sort order (Iceberg only marks files with a table-declared sort order).

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/SparkShufflingFileRewriteRunner.java:132

            .format("iceberg")
            .option(SparkReadOptions.SCAN_TASK_SET_ID, groupId)
            .load(groupId);

    Dataset<Row> sortedDF =
        sortedDF(
            scanDF,
            sortFunction(
                fileGroup.fileScanTasks(),
                spec(fileGroup.outputSpecId()),
                fileGroup.expectedOutputFiles()));

    org.apache.iceberg.SortOrder sortOrderInJobSpec = sortOrder();

    org.apache.iceberg.SortOrder maybeMatchingTableSortOrder =
        SortOrderUtil.findTableSortOrder(table(), sortOrder());

    if (sortOrderInJobSpec.isSorted() && maybeMatchingTableSortOrder.isUnsorted()) {
      LOG.warn(
          "Sort order specified for job {} doesn't match any table sort orders, rewritten files will not be marked as sorted in the manifest files",
          Spark3Util.describe(sortOrderInJobSpec));
    }

    sortedDF
        .write()
        .format("iceberg")
        .option(SparkWriteOptions.REWRITTEN_FILE_SCAN_TASK_SET_ID, groupId)
        .option(SparkWriteOptions.TARGET_FILE_SIZE_BYTES, fileGroup.maxOutputFileSize())
        .option(SparkWriteOptions.USE_TABLE_DISTRIBUTION_AND_ORDERING, "false")
        .option(SparkWriteOptions.OUTPUT_SPEC_ID, fileGroup.outputSpecId())
        .option(SparkWriteOptions.OUTPUT_SORT_ORDER_ID, maybeMatchingTableSortOrder.orderId())
        .mode("append")
        .save(groupId);
  }

  private Function<Dataset<Row>, Dataset<Row>> sortFunction(
      List<FileScanTask> group, PartitionSpec outputSpec, int expectedOutputFiles) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

Set the table sort order to match the job: ALTER TABLE t WRITE ORDERED BY col, or update sort order via UpdateSortOrder.
Change the job's sort option to exactly match an existing table sort order.
Accept the warning if physical sorting is all you need and you don't rely on sorted-manifest metadata for planning.

Example fix

// before
action.sort(asc("event_time")).execute(); // table has different/null sort order
// after: align table sort order first
table.updateSortOrder().addField("event_time").commit();
action.sort(asc("event_time")).execute();

Defensive patterns

Strategy: validation

Validate before calling

SortOrder tableOrder = table.sortOrder(); SortOrder jobOrder = /* SortOrder built from job spec */; boolean matches = SortOrderUtil.findTableSortOrder(table, jobOrder).isSorted();

Prevention

Set the table sort order (ALTER TABLE ... WRITE ORDERED BY) before sorting rewrites
Ensure job sort spec matches table order in field ids, direction, and null ordering
Don't rely on manifest sort metadata unless the warning is absent

When it happens

Trigger: Calling rewrite_data_files with sort_order/strategy options (e.g. via SparkActions...sort(asc("col"))) where SortOrderUtil.findTableSortOrder returns unsorted because the job's SortOrder isn't identical to any table sort order (field ids, direction, null ordering must all match).

Common situations: Specifying a sort on columns not covered by the table's sorted sort order; sort order set on the table after files were written; case/transform mismatch (identity vs truncate) between job spec and table spec; forgetting to first set the table sort order via ALTER TABLE ... ORDER BY.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/2beda41a500b5112. Report an issue: GitHub.