{"record":{"id":"7332150733459401","repo":"apache/iceberg","slug":"staging-snapshot-s-on-branch-s-removes-data-fi-733215","errorCode":null,"errorMessage":"Staging snapshot %s on branch '%s' removes data files; equality delete conversion does not support rewrites on the staging branch. Run compaction on the target branch instead.","messagePattern":"Staging snapshot (.+?) on branch '(.+?)' removes data files; equality delete conversion does not support rewrites on the staging branch\\. Run compaction on the target branch instead\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertPlanner.java","lineNumber":550,"sourceCode":"  /**\n   * Classifies the files added by {@code stagingSnapshot} into data files, eq delete files, and DV\n   * files. Throws if the snapshot:\n   *\n   * <ul>\n   *   <li>Removes data files (rewrites on the staging branch aren't supported).\n   *   <li>Contains V2 positional delete files (the converter expects a V3 staging branch written by\n   *       Flink, which produces only deletion vectors for deletes).\n   *   <li>Contains an eq-delete file whose {@code equalityFieldIds()} doesn't match the\n   *       builder-configured set (silent wrong-key serialization otherwise).\n   * </ul>\n   */\n  private StagingInputs retrieveStagingFiles(Snapshot stagingSnapshot) {\n    SnapshotChanges changes = SnapshotChanges.builderFor(table).snapshot(stagingSnapshot).build();\n\n    // Rewrites on the staging branch would require rewriting the corresponding DVs against new\n    // data files on target. Not implemented; fail fast instead of silently dropping work.\n    if (changes.removedDataFiles().iterator().hasNext()) {\n      throw new IllegalStateException(\n          String.format(\n              \"Staging snapshot %s on branch '%s' removes data files; \"\n                  + \"equality delete conversion does not support rewrites on the staging branch. \"\n                  + \"Run compaction on the target branch instead.\",\n              stagingSnapshot.snapshotId(), stagingBranch));\n    }\n\n    List<DataFile> newDataFiles = Lists.newArrayList();\n    List<DeleteFile> stagingDVFiles = Lists.newArrayList();\n    List<DeleteFile> eqDeleteFiles = Lists.newArrayList();\n\n    for (DataFile dataFile : changes.addedDataFiles()) {\n      newDataFiles.add(dataFile);\n    }\n\n    for (DeleteFile deleteFile : changes.addedDeleteFiles()) {\n      if (deleteFile.content() == FileContent.EQUALITY_DELETES) {\n        Set<Integer> deleteFieldIds = Sets.newHashSet(deleteFile.equalityFieldIds());","sourceCodeStart":532,"sourceCodeEnd":568,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertPlanner.java#L532-L568","documentation":"EqualityConvertPlanner.retrieveStagingFiles() inspects the staged snapshot's changes and rejects snapshots that removed (rewrote) data files. Equality delete conversion cannot rewrite the corresponding DVs against new data files, so it fails fast instead of silently dropping work, advising compaction be run on the target branch instead.","triggerScenarios":"The staging branch's latest snapshot contains a rewrite/replace commit (removedDataFiles non-empty) — e.g. compaction, file-rewrite, or expire/rewrite actions were run on the staging branch between conversion cycles.","commonSituations":"A separate compaction or rewrite job targeting the staging branch; user mistake running maintenance (rewrite_data_files) against the staging branch instead of main; automated optimization service rewriting small files on the wrong branch.","solutions":["Run compaction/rewrite jobs on the target (main) branch, not the staging branch","Rewind or reset the staging branch to a snapshot before the rewrite, then re-run conversion","Start a fresh staging branch for the conversion cycle","Adjust any automation so branch-targeted optimization excludes the staging branch"],"exampleFix":"// before\nCALL iceberg.system.rewrite_data_files(table => 'db.t', branch => 'staging')\n// after\nCALL iceberg.system.rewrite_data_files(table => 'db.t', branch => 'main')","handlingStrategy":"validation","validationCode":"// before running conversion, check staging snapshot for rewrites\nSnapshotChanges changes = SnapshotChanges.builderFor(table).snapshot(stagingSnapshot).build();\nboolean hasRewrites = changes.removedDataFiles().iterator().hasNext();\nif (hasRewrites) { /* reset staging branch or run compaction on main */ }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only run compaction/rewrite on the target branch","Exclude the staging branch from automated optimization services","Recreate the staging branch if it accumulated rewrites","Document branch roles so ops jobs target main"],"tags":["snapshot","rewrite","unsupported-operation","deletion-vector","flink"],"backgroundTag":"unsupported-operation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}