apache/iceberg · error · IllegalStateException

Main branch snapshot changed since planning: expected %s but

Error message

Main branch snapshot changed since planning: expected %s but found: %s

What it means

EqualityConvertDVWriter.resolveAndWrite compares the main branch snapshot it saw at planning time against the current snapshot when the watermark fires. If another writer (normal table commits, compaction, expiry) advanced the main branch in between, it throws IllegalStateException because deletion vectors written against the stale snapshot would be rejected by the committer's validateFromSnapshot check. Failing fast lets the next maintenance cycle reindex against the new snapshot.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertDVWriter.java:176

    super.processWatermark(mark);
  }

  private void resolveAndWrite() throws IOException {
    if (positionsByFile.isEmpty()) {
      return;
    }

    table.refresh();

    Snapshot mainSnapshot = table.snapshot(targetBranch);

    // Fail fast if the main branch changed since planning, to avoid writing DV files that the
    // committer would reject via validateFromSnapshot. The next cycle will reindex.
    if (mainSnapshot != null
        && planResult.mainSnapshotId() != null
        && mainSnapshot.snapshotId() != planResult.mainSnapshotId()) {
      throw new IllegalStateException(
          "Main branch snapshot changed since planning: expected "
              + planResult.mainSnapshotId()
              + " but found: "
              + mainSnapshot.snapshotId());
    }

    Map<String, DeleteFile> dvs = collectExistingDVs(mainSnapshot, positionsByFile.keySet());

    // Fold staging DVs into the rewrite so the writer emits one DV per data file (V3 rule). Flink
    // writes a staging DV only for a newly added data file, so it never collides with a distinct
    // existing DV: on a separate target branch collectExistingDVs has not seen it yet; on a shared
    // branch it IS that existing DV, so the put is idempotent.
    for (DeleteFile sd : planResult.stagingDVFiles()) {
      if (ContentFileUtil.isDV(sd) && sd.referencedDataFile() != null) {
        dvs.put(sd.referencedDataFile(), sd);
      }
    }

View on GitHub (pinned to 86d9c8fc54)

Solutions

Rerun the maintenance cycle — the converter re-plans against the new snapshot automatically.
Reduce concurrency: schedule the converter when no other writers commit to the main branch.
Shorten the plan-to-write window by tuning watermark generation gaps so planning is fresher.
If conflicts are frequent, isolate the table's maintenance on a dedicated branch or pause competing writers during the cycle.

Defensive patterns

Strategy: retry

Validate before calling

// before relying on a plan, check the branch hasn't moved
Snapshot current = table.currentSnapshot();
if (current != null && current.snapshotId() != planResult.mainSnapshotId()) {
  // re-plan before writing
}

Try / catch

try {
  dvWriter.resolveAndWrite(...);
} catch (IllegalStateException e) {
  if (e.getMessage().startsWith("Main branch snapshot changed")) {
    // schedule a fresh cycle; DVs were intentionally not written
    triggerReplan();
  } else {
    throw e;
  }
}

Prevention

Avoid concurrent committers to the same branch during conversion cycles.
Keep watermark gaps short so plans are recent when writes execute.
Retry cycles automatically — this error is safe to retry by design.
Monitor main-branch commit rate while maintenance runs.

When it happens

Trigger: processWatermark -> resolveAndWrite runs after a concurrent commit (e.g., a streaming ingest job or another maintenance task) changed the main branch snapshot between planResult creation and the watermark-driven write phase.

Common situations: Running the equality-delete converter concurrently with a Flink/Spark ingest job committing to the same branch; long watermark gaps that widen the plan-to-write window; expired-snapshot commits shifting the branch pointer.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ea4641f49fd0a0cb. Report an issue: GitHub.