apache/iceberg · error · ValidationException

Cannot apply non-unique WAP ID. Found multiple snapshots wit

Error message

Cannot apply non-unique WAP ID. Found multiple snapshots with WAP ID '%s'

What it means

PublishChangesProcedure cherry-picks a staged (WAP) snapshot identified by its write.wap.id property. ValidationException is thrown when more than one snapshot in the table carries the same WAP ID, because publishing would be ambiguous — the procedure refuses to guess which snapshot to publish.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/procedures/PublishChangesProcedure.java:102

  public ProcedureParameter[] parameters() {
    return PARAMETERS;
  }

  @Override
  public Iterator<Scan> call(InternalRow args) {
    ProcedureInput input = new ProcedureInput(spark(), tableCatalog(), PARAMETERS, args);

    Identifier tableIdent = input.ident(TABLE_PARAM);
    String wapId = input.asString(WAP_ID_PARAM);

    return modifyIcebergTable(
        tableIdent,
        table -> {
          Snapshot matchingSnap = null;
          for (Snapshot snap : table.snapshots()) {
            if (wapId.equals(WapUtil.stagedWapId(snap))) {
              if (matchingSnap != null) {
                throw new ValidationException(
                    "Cannot apply non-unique WAP ID. Found multiple snapshots with WAP ID '%s'",
                    wapId);
              } else {
                matchingSnap = snap;
              }
            }
          }

          if (matchingSnap == null) {
            throw new ValidationException("Cannot apply unknown WAP ID '%s'", wapId);
          }

          long wapSnapshotId = matchingSnap.snapshotId();
          table.manageSnapshots().cherrypick(wapSnapshotId).commit();
          Snapshot currentSnapshot = table.currentSnapshot();
          InternalRow outputRow = newInternalRow(wapSnapshotId, currentSnapshot.snapshotId());
          return asScanIterator(OUTPUT_TYPE, outputRow);
        });

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Generate a unique wap.id per write job (e.g. UUID) instead of reusing values
  2. Expire or delete the earlier duplicate staged snapshots, then retry publish_changes
  3. Cherry-pick the intended snapshot manually via table.manageSnapshots().cherrypick(snapshotId) to disambiguate
  4. List snapshots (SELECT * FROM table.snapshots) to find the duplicates before publishing

Example fix

// before
spark.conf.set("write.wap.id", "fixed-id"); // reused every run
// after
spark.conf.set("write.wap.id", java.util.UUID.randomUUID().toString());
Defensive patterns

Strategy: validation

Validate before calling

// detect duplicate WAP IDs before publishing
val dups = spark.table("cat.db.t.snapshots").where("properties[\"write.wap.id\"] = wapId").count()
if (dups > 1) fail("non-unique WAP ID")

Try / catch

try { spark.sql(call) } catch { case e: ValidationException if e.getMessage.contains("non-unique WAP ID") => /* expire duplicates or cherry-pick manually */ }

Prevention

When it happens

Trigger: Calling CALL cat.system.publish_changes(table => ..., id => 'X') when two or more staged snapshots were written with the same write.wap.id value (e.g. reusing a WAP ID across retries or jobs without expiring/cherry-picking earlier ones).

Common situations: Hard-coding the same wap.id in a CI pipeline across repeated runs; retrying a failed Spark write with the same wap.id after the first attempt actually succeeded as a staged snapshot; long-lived staged snapshots never published or expired.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/3e0d7c2a25fb64fb. Report an issue: GitHub.