apache/iceberg · error · IllegalStateException

Main branch snapshot changed since planning: expected {} but

Error message

Main branch snapshot changed since planning: expected {} but found: {}

What it means

EqualityConvertDVWriter.resolveAndWrite() compares the main branch snapshot observed at write time against the snapshot id captured during scan planning (planResult.mainSnapshotId()). If the main branch advanced meanwhile, it throws IllegalStateException to fail fast, because the committer would reject the produced deletion vector files via validateFromSnapshot anyway.

Source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertDVWriter.java:176

    super.processWatermark(mark);
  }

  private void resolveAndWrite() throws IOException {
    if (positionsByFile.isEmpty()) {
      return;
    }

    table.refresh();

    Snapshot mainSnapshot = table.snapshot(targetBranch);

    // Fail fast if the main branch changed since planning, to avoid writing DV files that the
    // committer would reject via validateFromSnapshot. The next cycle will reindex.
    if (mainSnapshot != null
        && planResult.mainSnapshotId() != null
        && mainSnapshot.snapshotId() != planResult.mainSnapshotId()) {
      throw new IllegalStateException(
          "Main branch snapshot changed since planning: expected "
              + planResult.mainSnapshotId()
              + " but found: "
              + mainSnapshot.snapshotId());
    }

    Map<String, DeleteFile> dvs = collectExistingDVs(mainSnapshot, positionsByFile.keySet());

    // Fold staging DVs into the rewrite so the writer emits one DV per data file (V3 rule). Flink
    // writes a staging DV only for a newly added data file, so it never collides with a distinct
    // existing DV: on a separate target branch collectExistingDVs has not seen it yet; on a shared
    // branch it IS that existing DV, so the put is idempotent.
    for (DeleteFile sd : planResult.stagingDVFiles()) {
      if (ContentFileUtil.isDV(sd) && sd.referencedDataFile() != null) {
        dvs.put(sd.referencedDataFile(), sd);
      }
    }

View on GitHub (pinned to 86d9c8fc54)

Solutions

Rerun the maintenance cycle — the next cycle reindexes from the new snapshot (this is the designed recovery)
Schedule maintenance during quiescent periods with fewer concurrent main-branch commits
Reduce cycle duration or increase trigger cadence so planning-to-write windows are short
Verify only expected writers commit to the main branch; route frequent commits to other branches if possible

Defensive patterns

Strategy: retry

Validate before calling

// before writing, re-check the current main snapshot matches planning
long current = table.refresh().currentSnapshot().snapshotId();
if (current != planResult.mainSnapshotId()) { /* skip and re-plan */ }

Try / catch

try { resolveAndWrite(...); } catch (IllegalStateException e) { if (e.getMessage().startsWith("Main branch snapshot changed")) { /* re-plan and re-run next cycle */ } else throw e; }

Prevention

Run DV reindexing when the main branch is quiet
Shorten the gap between scan planning and DV writing
Avoid overlapping maintenance cycles with heavy commit workloads
Route high-frequency commits away from the branch being reindexed

When it happens

Trigger: processWatermark -> resolveAndWrite() when a commit to the main table branch happened between DV scan planning and DV file writing in the same maintenance cycle.

Common situations: Concurrent writers committing to the main branch while maintenance reindexes; long-running maintenance cycle spanning normal table updates; frequent main-branch commits making reindex cycles repeatedly obsolete.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/b11a1bc73badc44f. Report an issue: GitHub.