{"record":{"id":"da9ad3248acad27e","repo":"apache/iceberg","slug":"found-conflicting-deleted-files-that-can-apply-to","errorCode":null,"errorMessage":"Found conflicting deleted files that can apply to records matching %s: %s","messagePattern":"Found conflicting deleted files that can apply to records matching (.+?): (.+?)","errorType":"validation","errorClass":"ValidationException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java","lineNumber":716,"sourceCode":"  }\n\n  /**\n   * Validates that no files matching a filter have been deleted from the table since a starting\n   * snapshot.\n   *\n   * @param base table metadata to validate\n   * @param startingSnapshotId id of the snapshot current at the start of the operation\n   * @param partitionSet a partition set used to find deleted data files\n   * @param parent ending snapshot on the branch being validated\n   */\n  protected void validateDeletedDataFiles(\n      TableMetadata base, Long startingSnapshotId, PartitionSet partitionSet, Snapshot parent) {\n    CloseableIterable<ManifestEntry<DataFile>> conflictEntries =\n        deletedDataFiles(base, startingSnapshotId, null, partitionSet, parent);\n\n    try (CloseableIterator<ManifestEntry<DataFile>> conflicts = conflictEntries.iterator()) {\n      if (conflicts.hasNext()) {\n        throw new ValidationException(\n            \"Found conflicting deleted files that can apply to records matching %s: %s\",\n            partitionSet,\n            Iterators.toString(\n                Iterators.transform(conflicts, entry -> entry.file().location().toString())));\n      }\n\n    } catch (IOException e) {\n      throw new UncheckedIOException(\n          String.format(\"Failed to validate no appends matching %s\", partitionSet), e);\n    }\n  }\n\n  /**\n   * Returns an iterable of files matching a filter have been added to the table since a starting\n   * snapshot.\n   *\n   * @param base table metadata to validate\n   * @param startingSnapshotId id of the snapshot current at the start of the operation","sourceCodeStart":698,"sourceCodeEnd":734,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java#L698-L734","documentation":"ValidationException thrown at commit when a concurrent snapshot deleted data files whose partitions fall within the provided PartitionSet, meaning those deletes can apply to records the current operation touches. Iceberg rejects the commit to preserve correctness under concurrent modifications. Like other validation failures, rebasing on the latest snapshot is required.","triggerScenarios":"Calling validateNoDeletedFiles with a PartitionSet (e.g., from BaseOverwriteFiles.validate or RowDelta operations) while concurrent commits deleted data files in overlapping partitions since the starting snapshot.","commonSituations":"Concurrent partition-level rewrites (compaction) racing with row-level deletes/overwrites; Spark MERGE/DELETE jobs overlapping with rewriteDataFiles procedures; multi-writer pipelines without coordination.","solutions":["Refresh the table and rebase the operation on the current snapshot, then recompute the PartitionSet and retry.","Sequence conflicting jobs (compaction vs delete/merge) so they don't overlap on the same partitions.","Restrict the operation to partitions not concurrently being compacted/rewritten.","Use branch-based isolation (WAP branches) to stage and fast-forward atomically."],"exampleFix":"// before\nrowDelta.validateNoDeletedFiles(startSnapshotId, conflictingPartitions).commit(); // throws on concurrent compaction\n// after\ntable.refresh();\nRowDelta op = table.newRowDelta();\nop.validateNoDeletedFiles(table.currentSnapshot().snapshotId(), conflictingPartitions);\nop.commit(); // retry after rebase","handlingStrategy":"retry","validationCode":"table.refresh(); // recompute partition set against current state\nPartitionSet set = PartitionSet.create(table.specs());","typeGuard":null,"tryCatchPattern":"try {\n  operation.commit();\n} catch (ValidationException e) {\n  table.refresh();\n  // rebase operation and recompute the PartitionSet, then retry\n}","preventionTips":["Sequence compaction and DML on the same partitions","Refresh before validating and committing","Narrow operations to partitions not under concurrent rewrite"],"tags":["concurrency","optimistic-locking","partition-conflict","commit-validation"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}