apache/iceberg · error · CommitFailedException

Requirement failed: last assigned field id changed: expected

Error message

Requirement failed: last assigned field id changed: expected id %s != %s

What it means

Thrown by AssertLastAssignedFieldId.validate when the requirement's lastAssignedFieldId does not match the base table's lastColumnId (the highest assigned nested-field ID). Field IDs are monotonic and must never be reused; this assertion guarantees no concurrent schema change assigned new field IDs between planning and commit. A mismatch means another writer added/dropped columns concurrently, so the commit fails.

Source

Thrown at core/src/main/java/org/apache/iceberg/UpdateRequirement.java:143

      }
    }
  }

  class AssertLastAssignedFieldId implements UpdateRequirement {
    private final int lastAssignedFieldId;

    public AssertLastAssignedFieldId(int lastAssignedFieldId) {
      this.lastAssignedFieldId = lastAssignedFieldId;
    }

    public int lastAssignedFieldId() {
      return lastAssignedFieldId;
    }

    @Override
    public void validate(TableMetadata base) {
      if (base != null && base.lastColumnId() != lastAssignedFieldId) {
        throw new CommitFailedException(
            "Requirement failed: last assigned field id changed: expected id %s != %s",
            lastAssignedFieldId, base.lastColumnId());
      }
    }
  }

  class AssertCurrentSchemaID implements UpdateRequirement {
    private final int schemaId;

    public AssertCurrentSchemaID(int schemaId) {
      this.schemaId = schemaId;
    }

    public int schemaId() {
      return schemaId;
    }

    @Override

View on GitHub (pinned to 86d9c8fc54)

Solutions

Refresh the table, re-read lastColumnId, rebuild requirements and re-apply the schema change, then retry the commit.
Re-apply the schema update on top of the refreshed schema so new field IDs are assigned correctly (never reuse asserted IDs).
Coordinate schema evolution through a single owner or migration tool to avoid concurrent column additions.
Catch CommitFailedException and retry with backoff, as Iceberg commits are designed to be retried against fresh metadata.

Example fix

// before
List<UpdateRequirement> reqs = ImmutableList.of(
    new UpdateRequirement.AssertLastAssignedFieldId(42)); // stale
// after
table.refresh();
List<UpdateRequirement> reqs = ImmutableList.of(
    new UpdateRequirement.AssertLastAssignedFieldId(table.schema().highestFieldId()));

Defensive patterns

Strategy: retry

Validate before calling

table.refresh();
if (table.schema().highestFieldId() != assertedFieldId) {
  // rebuild requirements from current metadata before committing
}

Try / catch

try {
  schemaUpdate.commit();
} catch (CommitFailedException e) {
  if (e.getMessage().contains("last assigned field id changed")) {
    table.refresh();
    reapplySchemaChange(table); // retry with backoff
  } else { throw e; }
}

Prevention

Route all schema evolution through one owner/pipeline
Refresh the table before schema updates
Never reuse hardcoded field IDs; derive from current schema
Implement CommitFailedException retry with backoff for all metadata commits

When it happens

Trigger: Committing schema updates with UpdateRequirements including AssertLastAssignedFieldId when base.lastColumnId() differs from the asserted value — any concurrent AddColumn/UpdateColumn operation on the table invalidates the assertion.

Common situations: Two jobs evolve the schema simultaneously (e.g. a streaming pipeline adding columns while a batch job alters types); REST catalog commits from a stale client-side cached table; schema-drift tools (e.g. schema registries auto-adding fields) racing with manual ALTER TABLE.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/380fb62c08960f07. Report an issue: GitHub.