apache/iceberg · error · ValidationException

Cannot apply non-unique WAP ID. Found multiple snapshots wit

Error message

Cannot apply non-unique WAP ID. Found multiple snapshots with WAP ID '%s'

What it means

PublishChangesProcedure cherry-picks a staged (write-audit-publish) snapshot identified by its 'wap.id' snapshot summary property. Before publishing, it scans all snapshots for the given WAP ID; if two or more staged snapshots carry the same ID, it aborts, because it cannot determine which snapshot to publish. This guards against reusing a WAP ID across commits.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/procedures/PublishChangesProcedure.java:102

  public ProcedureParameter[] parameters() {
    return PARAMETERS;
  }

  @Override
  public Iterator<Scan> call(InternalRow args) {
    ProcedureInput input = new ProcedureInput(spark(), tableCatalog(), PARAMETERS, args);

    Identifier tableIdent = input.ident(TABLE_PARAM);
    String wapId = input.asString(WAP_ID_PARAM);

    return modifyIcebergTable(
        tableIdent,
        table -> {
          Snapshot matchingSnap = null;
          for (Snapshot snap : table.snapshots()) {
            if (wapId.equals(WapUtil.stagedWapId(snap))) {
              if (matchingSnap != null) {
                throw new ValidationException(
                    "Cannot apply non-unique WAP ID. Found multiple snapshots with WAP ID '%s'",
                    wapId);
              } else {
                matchingSnap = snap;
              }
            }
          }

          if (matchingSnap == null) {
            throw new ValidationException("Cannot apply unknown WAP ID '%s'", wapId);
          }

          long wapSnapshotId = matchingSnap.snapshotId();
          table.manageSnapshots().cherrypick(wapSnapshotId).commit();
          Snapshot currentSnapshot = table.currentSnapshot();
          InternalRow outputRow = newInternalRow(wapSnapshotId, currentSnapshot.snapshotId());
          return asScanIterator(OUTPUT_TYPE, outputRow);
        });

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Ensure each staged write gets a unique WAP ID (per-run UUID), e.g. derive spark.wap.id per execution.
  2. Expire or delete the stale staged snapshots so only one carries the ID, then retry publish.
  3. Publish the specific snapshot explicitly instead: cherry-pick the desired snapshotId via manageSnapshots/rewrite the procedure call.
  4. Audit the writer configuration generating wap.id for reuse.

Example fix

// before (reused static id)
spark.conf.set("spark.wap.id", "run-42");
// after (unique per run)
spark.conf.set("spark.wap.id", java.util.UUID.randomUUID().toString());
Defensive patterns

Strategy: validation

Validate before calling

SELECT snapshot_id, summary['wap.id'] FROM db.t.snapshots WHERE summary['wap.id'] = 'my-id'; // expect exactly one row

Try / catch

try { publishChanges(table, wapId); } catch (ValidationException e) { /* duplicate WAP ID: expire stale snapshot or use unique IDs */ }

Prevention

When it happens

Trigger: Calling CALL iceberg.system.publish_changes(table => 'db.t', wap_id => 'some-id') when the table's history contains two staged snapshots both stamped with that same wap.id — typically because a writer reused a fixed wap.id value across multiple writes without publishing in between.

Common situations: A Spark job sets spark.wap.id to a static value in config instead of a per-run UUID; two concurrent/staged branches of the same job write with the same WAP ID; a failed publish followed by a re-run staged a second snapshot with the same ID.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5710ccbc41cb3340. Report an issue: GitHub.