{"record":{"id":"b11a1bc73badc44f","repo":"apache/iceberg","slug":"main-branch-snapshot-changed-since-planning-expec-b11a1b","errorCode":null,"errorMessage":"Main branch snapshot changed since planning: expected {} but found: {}","messagePattern":"Main branch snapshot changed since planning: expected (.+?) but found: (.+?)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertDVWriter.java","lineNumber":176,"sourceCode":"\n    super.processWatermark(mark);\n  }\n\n  private void resolveAndWrite() throws IOException {\n    if (positionsByFile.isEmpty()) {\n      return;\n    }\n\n    table.refresh();\n\n    Snapshot mainSnapshot = table.snapshot(targetBranch);\n\n    // Fail fast if the main branch changed since planning, to avoid writing DV files that the\n    // committer would reject via validateFromSnapshot. The next cycle will reindex.\n    if (mainSnapshot != null\n        && planResult.mainSnapshotId() != null\n        && mainSnapshot.snapshotId() != planResult.mainSnapshotId()) {\n      throw new IllegalStateException(\n          \"Main branch snapshot changed since planning: expected \"\n              + planResult.mainSnapshotId()\n              + \" but found: \"\n              + mainSnapshot.snapshotId());\n    }\n\n    Map<String, DeleteFile> dvs = collectExistingDVs(mainSnapshot, positionsByFile.keySet());\n\n    // Fold staging DVs into the rewrite so the writer emits one DV per data file (V3 rule). Flink\n    // writes a staging DV only for a newly added data file, so it never collides with a distinct\n    // existing DV: on a separate target branch collectExistingDVs has not seen it yet; on a shared\n    // branch it IS that existing DV, so the put is idempotent.\n    for (DeleteFile sd : planResult.stagingDVFiles()) {\n      if (ContentFileUtil.isDV(sd) && sd.referencedDataFile() != null) {\n        dvs.put(sd.referencedDataFile(), sd);\n      }\n    }\n","sourceCodeStart":158,"sourceCodeEnd":194,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertDVWriter.java#L158-L194","documentation":"EqualityConvertDVWriter.resolveAndWrite() compares the main branch snapshot observed at write time against the snapshot id captured during scan planning (planResult.mainSnapshotId()). If the main branch advanced meanwhile, it throws IllegalStateException to fail fast, because the committer would reject the produced deletion vector files via validateFromSnapshot anyway.","triggerScenarios":"processWatermark -> resolveAndWrite() when a commit to the main table branch happened between DV scan planning and DV file writing in the same maintenance cycle.","commonSituations":"Concurrent writers committing to the main branch while maintenance reindexes; long-running maintenance cycle spanning normal table updates; frequent main-branch commits making reindex cycles repeatedly obsolete.","solutions":["Rerun the maintenance cycle — the next cycle reindexes from the new snapshot (this is the designed recovery)","Schedule maintenance during quiescent periods with fewer concurrent main-branch commits","Reduce cycle duration or increase trigger cadence so planning-to-write windows are short","Verify only expected writers commit to the main branch; route frequent commits to other branches if possible"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// before writing, re-check the current main snapshot matches planning\nlong current = table.refresh().currentSnapshot().snapshotId();\nif (current != planResult.mainSnapshotId()) { /* skip and re-plan */ }","typeGuard":null,"tryCatchPattern":"try { resolveAndWrite(...); } catch (IllegalStateException e) { if (e.getMessage().startsWith(\"Main branch snapshot changed\")) { /* re-plan and re-run next cycle */ } else throw e; }","preventionTips":["Run DV reindexing when the main branch is quiet","Shorten the gap between scan planning and DV writing","Avoid overlapping maintenance cycles with heavy commit workloads","Route high-frequency commits away from the branch being reindexed"],"tags":["deletion-vector","snapshot-conflict","concurrency","flink"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}