{"record":{"id":"920c5ab0c88239e5","repo":"apache/iceberg","slug":"cannot-delete-data-files-s-that-are-referenced-by","errorCode":null,"errorMessage":"Cannot delete data files %s that are referenced by new delete files","messagePattern":"Cannot delete data files (.+?) that are referenced by new delete files","errorType":"validation","errorClass":"ValidationException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/BaseRowDelta.java","lineNumber":189,"sourceCode":"\n      validateAddedDVs(base, startingSnapshotId, conflictDetectionFilter, parent);\n    }\n  }\n\n  /**\n   * Validates that the data files removed in this commit do not overlap with data files with delete\n   * files added\n   */\n  @SuppressWarnings(\"CollectionUndefinedEquality\")\n  private void validateNoConflictingFileAndPositionDeletes() {\n    List<CharSequence> deletedFileWithNewDVs =\n        removedDataFiles.stream()\n            .map(DataFile::path)\n            .filter(referencedDataFiles::contains)\n            .collect(Collectors.toList());\n\n    if (!deletedFileWithNewDVs.isEmpty()) {\n      throw new ValidationException(\n          \"Cannot delete data files %s that are referenced by new delete files\",\n          deletedFileWithNewDVs);\n    }\n  }\n}\n","sourceCodeStart":171,"sourceCodeEnd":195,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/BaseRowDelta.java#L171-L195","documentation":"RowDelta operations both append delete (position/deletion-vector) files and can delete data files; this validation rejects commits where a data file being deleted is referenced by a delete file in the same operation. Deleting the data file would orphan the new delete file's positions, corrupting row-level deletion semantics.","triggerScenarios":"Calling newRowDelta() that both (a) addsDeleteFile / DV writers referencing data files and (b) deletes data files with deleteRows/deleteDataFiles for those same paths in one commit.","commonSituations":"Compaction jobs that remove rewritten data files while a concurrent merge-on-read writer publishes delete files targeting them; misconfigured maintenance pipelines issuing both changes in one RowDelta instead of separate snapshots.","solutions":["Split the work: commit the RowDelta that adds delete files first, then delete the data files in a separate, later operation (e.g. via deleteFiles/rewriteDataFiles)","Refresh the table and re-validate that targeted data file paths are not in newly added delete files before deleting","Use rewriteDataFiles instead of manual RowDelta so compaction and deletes are coordinated","Coordinate merge-on-read writers and compaction jobs to avoid racing on the same files"],"exampleFix":"// before: same RowDelta does both\nTransaction tx = table.newTransaction();\ntable.newRowDelta().addDeletes(deleteFile).deleteDataFile(dataFile).commit();\n// after: sequence operations\ntable.newRowDelta().addDeletes(deleteFile).commit(); // snapshot A\ntable.refresh();\ntable.newDelete().deleteFile(dataFile).commit(); // snapshot B","handlingStrategy":"validation","validationCode":"Set<CharSequence> referenced = newDeleteFiles.stream().flatMap(df -> df.referencedDataFiles().stream()).collect(Collectors.toSet());\nboolean conflict = dataFilesToDelete.stream().anyMatch(f -> referenced.contains(f.path()));\nif (conflict) throw new IllegalStateException(\"data files referenced by new delete files\");","typeGuard":null,"tryCatchPattern":"try { rowDelta.commit(); } catch (ValidationException e) { table.refresh(); reapplyAsSeparateCommits(); }","preventionTips":["Never combine addDeletes with deleteDataFile for the same data files in one RowDelta","Sequence DV publishing before compaction/file deletion","Use high-level rewriteDataFiles APIs instead of manual RowDelta for maintenance"],"tags":["iceberg","row-delta","conflict-validation","delete-files"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}