apache/iceberg · error · CommitFailedException

Requirement failed: default partition spec changed: expected

Error message

Requirement failed: default partition spec changed: expected id %s != %s

What it means

Thrown by AssertDefaultSpecId.validate when the requirement's specId differs from base.defaultSpecId(). This requirement asserts the table's default partition spec is unchanged since the update was planned. A mismatch means a concurrent writer switched the default spec (partition evolution), so the commit is rejected to prevent writing data under assumptions about the wrong spec.

Source

Thrown at core/src/main/java/org/apache/iceberg/UpdateRequirement.java:206

      }
    }
  }

  class AssertDefaultSpecID implements UpdateRequirement {
    private final int specId;

    public AssertDefaultSpecID(int specId) {
      this.specId = specId;
    }

    public int specId() {
      return specId;
    }

    @Override
    public void validate(TableMetadata base) {
      if (specId != base.defaultSpecId()) {
        throw new CommitFailedException(
            "Requirement failed: default partition spec changed: expected id %s != %s",
            specId, base.defaultSpecId());
      }
    }
  }

  class AssertDefaultSortOrderID implements UpdateRequirement {
    private final int sortOrderId;

    public AssertDefaultSortOrderID(int sortOrderId) {
      this.sortOrderId = sortOrderId;
    }

    public int sortOrderId() {
      return sortOrderId;
    }

    @Override

View on GitHub (pinned to 86d9c8fc54)

Solutions

Refresh metadata, read base.defaultSpecId(), rebuild the requirements, and retry the commit.
Re-evaluate the operation under the new default spec — writes planned for the old spec may need re-partitioning before retry.
Gate partition-evolution changes behind a single coordinated job so concurrent default-spec switches cannot occur.
Catch CommitFailedException and retry the update against fresh metadata with backoff.

Example fix

// before
UpdateRequirement req = new UpdateRequirement.AssertDefaultSpecId(0); // default now spec 1
// after
table.refresh();
UpdateRequirement req = new UpdateRequirement.AssertDefaultSpecId(
    table.spec().specId());

Defensive patterns

Strategy: retry

Validate before calling

table.refresh();
if (table.spec().specId() != expectedDefaultSpecId) {
  throw new IllegalStateException("Default spec changed; re-plan writes");
}

Try / catch

try {
  commitWithRequirements(reqs);
} catch (CommitFailedException e) {
  if (e.getMessage().contains("default partition spec changed")) {
    table.refresh();
    reqs = rebuildRequirements(table);
  } else { throw e; }
}

Prevention

Check defaultSpecId before writes that assume a particular spec
Coordinate partition-evolution migrations with active writers
Refresh the table before each commit batch
Implement standard CommitFailedException retry/backoff

When it happens

Trigger: Committing updates with AssertDefaultSpecId when base.defaultSpecId() != specId — another job ran UpdatePartitionSpec (replacing the default spec) or reset the default spec concurrently.

Common situations: A partition-evolution migration running while writers commit other metadata changes; two admin jobs both altering partitioning; commits from a client whose cached metadata predates a spec change; REST catalog conflicts between independent services managing the table.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ebe763eb0c5ca4ef. Report an issue: GitHub.