{"record":{"id":"3603aa261f1c7be6","repo":"apache/iceberg","slug":"found-conflicting-deleted-files-that-can-contain-r","errorCode":null,"errorMessage":"Found conflicting deleted files that can contain records matching %s: %s","messagePattern":"Found conflicting deleted files that can contain records matching (.+?): (.+?)","errorType":"validation","errorClass":"ValidationException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java","lineNumber":687,"sourceCode":"  }\n\n  /**\n   * Validates that no files matching a filter have been deleted from the table since a starting\n   * snapshot.\n   *\n   * @param base table metadata to validate\n   * @param startingSnapshotId id of the snapshot current at the start of the operation\n   * @param dataFilter an expression used to find deleted data files\n   * @param parent ending snapshot on the branch being validated\n   */\n  protected void validateDeletedDataFiles(\n      TableMetadata base, Long startingSnapshotId, Expression dataFilter, Snapshot parent) {\n    CloseableIterable<ManifestEntry<DataFile>> conflictEntries =\n        deletedDataFiles(base, startingSnapshotId, dataFilter, null, parent);\n\n    try (CloseableIterator<ManifestEntry<DataFile>> conflicts = conflictEntries.iterator()) {\n      if (conflicts.hasNext()) {\n        throw new ValidationException(\n            \"Found conflicting deleted files that can contain records matching %s: %s\",\n            dataFilter,\n            Iterators.toString(\n                Iterators.transform(conflicts, entry -> entry.file().location().toString())));\n      }\n\n    } catch (IOException e) {\n      throw new UncheckedIOException(\n          String.format(\"Failed to validate no deleted data files matching %s\", dataFilter), e);\n    }\n  }\n\n  /**\n   * Validates that no files matching a filter have been deleted from the table since a starting\n   * snapshot.\n   *\n   * @param base table metadata to validate\n   * @param startingSnapshotId id of the snapshot current at the start of the operation","sourceCodeStart":669,"sourceCodeEnd":705,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java#L669-L705","documentation":"ValidationException thrown during commit when another concurrent snapshot deleted data files that could contain records matching the given filter since the starting snapshot. Iceberg aborts the commit because proceeding could lose or corrupt data affected by those concurrent deletes. This is a genuine optimistic-concurrency conflict, not an I/O bug.","triggerScenarios":"Committing an operation (e.g., overwrite/delete) that calls validateNoDeletedFiles with a data filter while a concurrent committed snapshot deleted data files matching that filter on the same branch.","commonSituations":"Two writers overwriting the same partition concurrently; a compaction/expiry job deleting files while another job rewrites them; long-running streaming commits overlapping with delete operations.","solutions":["Refresh the table (table.refresh()), rebase the operation on the current snapshot, and re-apply the change.","Retry the whole operation after refresh; only one writer's change will win per snapshot.","Reduce conflict scope by narrowing the data filter/partition so concurrent deletes in other partitions don't clash.","Coordinate writers (external locking or single-writer pattern) for heavily contended tables."],"exampleFix":"// before\ntable.newOverwrite().overwriteByRowFilter(filter).validateNoDeletedFiles(startSnapshotId).commit(); // may throw\n// after\ntable.refresh();\nOverwriteFiles op = table.newOverwrite().overwriteByRowFilter(filter);\nop.validateNoDeletedFiles(table.currentSnapshot().snapshotId()); // validate against latest snapshot\nop.commit();","handlingStrategy":"retry","validationCode":"table.refresh(); // rebase on latest snapshot before committing\nSnapshot current = table.currentSnapshot();","typeGuard":null,"tryCatchPattern":"try {\n  operation.commit();\n} catch (ValidationException e) {\n  table.refresh();\n  // rebuild and re-apply the operation on the current snapshot, then retry\n}","preventionTips":["Always refresh() and rebase operations before commit","Serialize contended jobs (compaction vs deletes) on the same partitions","Use separate branches/WAP for isolating concurrent writers"],"tags":["concurrency","optimistic-locking","conflict","commit-validation"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}