{"record":{"id":"ea4641f49fd0a0cb","repo":"apache/iceberg","slug":"main-branch-snapshot-changed-since-planning-expec-ea4641","errorCode":null,"errorMessage":"Main branch snapshot changed since planning: expected %s but found: %s","messagePattern":"Main branch snapshot changed since planning: expected (.+?) but found: (.+?)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertDVWriter.java","lineNumber":176,"sourceCode":"\n    super.processWatermark(mark);\n  }\n\n  private void resolveAndWrite() throws IOException {\n    if (positionsByFile.isEmpty()) {\n      return;\n    }\n\n    table.refresh();\n\n    Snapshot mainSnapshot = table.snapshot(targetBranch);\n\n    // Fail fast if the main branch changed since planning, to avoid writing DV files that the\n    // committer would reject via validateFromSnapshot. The next cycle will reindex.\n    if (mainSnapshot != null\n        && planResult.mainSnapshotId() != null\n        && mainSnapshot.snapshotId() != planResult.mainSnapshotId()) {\n      throw new IllegalStateException(\n          \"Main branch snapshot changed since planning: expected \"\n              + planResult.mainSnapshotId()\n              + \" but found: \"\n              + mainSnapshot.snapshotId());\n    }\n\n    Map<String, DeleteFile> dvs = collectExistingDVs(mainSnapshot, positionsByFile.keySet());\n\n    // Fold staging DVs into the rewrite so the writer emits one DV per data file (V3 rule). Flink\n    // writes a staging DV only for a newly added data file, so it never collides with a distinct\n    // existing DV: on a separate target branch collectExistingDVs has not seen it yet; on a shared\n    // branch it IS that existing DV, so the put is idempotent.\n    for (DeleteFile sd : planResult.stagingDVFiles()) {\n      if (ContentFileUtil.isDV(sd) && sd.referencedDataFile() != null) {\n        dvs.put(sd.referencedDataFile(), sd);\n      }\n    }\n","sourceCodeStart":158,"sourceCodeEnd":194,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertDVWriter.java#L158-L194","documentation":"EqualityConvertDVWriter.resolveAndWrite compares the main branch snapshot it saw at planning time against the current snapshot when the watermark fires. If another writer (normal table commits, compaction, expiry) advanced the main branch in between, it throws IllegalStateException because deletion vectors written against the stale snapshot would be rejected by the committer's validateFromSnapshot check. Failing fast lets the next maintenance cycle reindex against the new snapshot.","triggerScenarios":"processWatermark -> resolveAndWrite runs after a concurrent commit (e.g., a streaming ingest job or another maintenance task) changed the main branch snapshot between planResult creation and the watermark-driven write phase.","commonSituations":"Running the equality-delete converter concurrently with a Flink/Spark ingest job committing to the same branch; long watermark gaps that widen the plan-to-write window; expired-snapshot commits shifting the branch pointer.","solutions":["Rerun the maintenance cycle — the converter re-plans against the new snapshot automatically.","Reduce concurrency: schedule the converter when no other writers commit to the main branch.","Shorten the plan-to-write window by tuning watermark generation gaps so planning is fresher.","If conflicts are frequent, isolate the table's maintenance on a dedicated branch or pause competing writers during the cycle."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// before relying on a plan, check the branch hasn't moved\nSnapshot current = table.currentSnapshot();\nif (current != null && current.snapshotId() != planResult.mainSnapshotId()) {\n  // re-plan before writing\n}","typeGuard":null,"tryCatchPattern":"try {\n  dvWriter.resolveAndWrite(...);\n} catch (IllegalStateException e) {\n  if (e.getMessage().startsWith(\"Main branch snapshot changed\")) {\n    // schedule a fresh cycle; DVs were intentionally not written\n    triggerReplan();\n  } else {\n    throw e;\n  }\n}","preventionTips":["Avoid concurrent committers to the same branch during conversion cycles.","Keep watermark gaps short so plans are recent when writes execute.","Retry cycles automatically — this error is safe to retry by design.","Monitor main-branch commit rate while maintenance runs."],"tags":["concurrency","snapshot","flink","conflict"],"backgroundTag":"concurrent-snapshot-modification","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}