{"record":{"id":"bb22cdb8668b1cf6","repo":"apache/iceberg","slug":"staging-snapshot-s-on-branch-s-removes-data-fi-bb22cd","errorCode":null,"errorMessage":"Staging snapshot %s on branch '%s' removes data files; equality delete conversion does not support rewrites on the staging branch. Run compaction on the target branch instead.","messagePattern":"Staging snapshot (.+?) on branch '(.+?)' removes data files; equality delete conversion does not support rewrites on the staging branch\\. Run compaction on the target branch instead\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertPlanner.java","lineNumber":550,"sourceCode":"  /**\n   * Classifies the files added by {@code stagingSnapshot} into data files, eq delete files, and DV\n   * files. Throws if the snapshot:\n   *\n   * <ul>\n   *   <li>Removes data files (rewrites on the staging branch aren't supported).\n   *   <li>Contains V2 positional delete files (the converter expects a V3 staging branch written by\n   *       Flink, which produces only deletion vectors for deletes).\n   *   <li>Contains an eq-delete file whose {@code equalityFieldIds()} doesn't match the\n   *       builder-configured set (silent wrong-key serialization otherwise).\n   * </ul>\n   */\n  private StagingInputs retrieveStagingFiles(Snapshot stagingSnapshot) {\n    SnapshotChanges changes = SnapshotChanges.builderFor(table).snapshot(stagingSnapshot).build();\n\n    // Rewrites on the staging branch would require rewriting the corresponding DVs against new\n    // data files on target. Not implemented; fail fast instead of silently dropping work.\n    if (changes.removedDataFiles().iterator().hasNext()) {\n      throw new IllegalStateException(\n          String.format(\n              \"Staging snapshot %s on branch '%s' removes data files; \"\n                  + \"equality delete conversion does not support rewrites on the staging branch. \"\n                  + \"Run compaction on the target branch instead.\",\n              stagingSnapshot.snapshotId(), stagingBranch));\n    }\n\n    List<DataFile> newDataFiles = Lists.newArrayList();\n    List<DeleteFile> stagingDVFiles = Lists.newArrayList();\n    List<DeleteFile> eqDeleteFiles = Lists.newArrayList();\n\n    for (DataFile dataFile : changes.addedDataFiles()) {\n      newDataFiles.add(dataFile);\n    }\n\n    for (DeleteFile deleteFile : changes.addedDeleteFiles()) {\n      if (deleteFile.content() == FileContent.EQUALITY_DELETES) {\n        Set<Integer> deleteFieldIds = Sets.newHashSet(deleteFile.equalityFieldIds());","sourceCodeStart":532,"sourceCodeEnd":568,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertPlanner.java#L532-L568","documentation":"Thrown by EqualityConvertPlanner.retrieveStagingFiles when the staging-branch snapshot contains removed data files, i.e. a rewrite (compaction) happened on the staging branch. Since the converter cannot rewrite the corresponding DVs against the new data files on the target branch, it fails fast instead of silently dropping work.","triggerScenarios":"A snapshot staged on stagingBranch was produced by a rewriteDataFiles/compaction job (or any operation that removes data files), and then the equality-delete-conversion planner inspects that snapshot via SnapshotChanges.removedDataFiles().","commonSituations":"Scheduling compaction on the maintenance staging branch by mistake; sharing one branch for both compaction and delete conversion in the maintenance configuration; an automated rewrite policy targeting the wrong branch.","solutions":["Run compaction (rewriteDataFiles) on the target branch, not the staging branch, before or after conversion.","Point the maintenance table's staging branch config at a branch used only for staged delete commits.","Rewind or recreate the staging branch to a snapshot without data-file rewrites and rerun the conversion job."],"exampleFix":"// before: compaction on staging branch\nRewriteDataFilesAction.forTable(table).onBranch(\"maintenance-staging\").execute();\n// after: compaction on target branch\nRewriteDataFilesAction.forTable(table).onBranch(targetBranch).execute();","handlingStrategy":"validation","validationCode":"Snapshot staging = table.snapshotForBranch(stagingBranch);\nSnapshotChanges changes = SnapshotChanges.builderFor(table).snapshot(staging).build();\nif (changes.removedDataFiles().iterator().hasNext()) {\n  throw new IllegalStateException(\"Staging branch has rewrites; run compaction on target branch instead\");\n}\n// else safe to run conversion","typeGuard":null,"tryCatchPattern":"try {\n  runEqualityConvertJob(table, cfg);\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"removes data files\")) {\n    LOG.warn(\"Staging branch has compaction rewrites; redirect compaction to target branch\", e);\n  } else throw e;\n}","preventionTips":["Only use the staging branch for staged delete commits","Schedule compaction (rewriteDataFiles) on the target branch","Review any automated rewrite policies for branch targeting","Validate the staging snapshot before conversion with SnapshotChanges"],"tags":["flink","compaction","branching","precondition"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}