{"record":{"id":"927ba87895215ca7","repo":"apache/iceberg","slug":"found-conflicting-files-that-can-contain-records-m-927ba8","errorCode":null,"errorMessage":"Found conflicting files that can contain records matching %s: %s","messagePattern":"Found conflicting files that can contain records matching (.+?): (.+?)","errorType":"validation","errorClass":"ValidationException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java","lineNumber":401,"sourceCode":"  /**\n   * Validates that no files matching a filter have been added to the table since a starting\n   * snapshot.\n   *\n   * @param base table metadata to validate\n   * @param startingSnapshotId id of the snapshot current at the start of the operation\n   * @param conflictDetectionFilter an expression used to find new conflicting data files\n   */\n  protected void validateAddedDataFiles(\n      TableMetadata base,\n      Long startingSnapshotId,\n      Expression conflictDetectionFilter,\n      Snapshot parent) {\n    CloseableIterable<ManifestEntry<DataFile>> conflictEntries =\n        addedDataFiles(base, startingSnapshotId, conflictDetectionFilter, null, parent);\n\n    try (CloseableIterator<ManifestEntry<DataFile>> conflicts = conflictEntries.iterator()) {\n      if (conflicts.hasNext()) {\n        throw new ValidationException(\n            \"Found conflicting files that can contain records matching %s: %s\",\n            conflictDetectionFilter,\n            Iterators.toString(\n                Iterators.transform(conflicts, entry -> entry.file().location().toString())));\n      }\n\n    } catch (IOException e) {\n      throw new UncheckedIOException(\n          String.format(\"Failed to validate no appends matching %s\", conflictDetectionFilter), e);\n    }\n  }\n\n  /**\n   * Returns an iterable of files matching a filter have been added to a branch since a starting\n   * snapshot.\n   *\n   * @param base table metadata to validate\n   * @param startingSnapshotId id of the snapshot current at the start of the operation","sourceCodeStart":383,"sourceCodeEnd":419,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java#L383-L419","documentation":"Filter-based variant of the conflicting-files validation. validateAddedDataFiles scans data files added since the starting snapshot that match the given conflictDetectionFilter; if any exist, it throws this ValidationException listing the filter and the conflicting file locations. This prevents commits (e.g. deletes/overwrites) from being based on data that has since changed.","triggerScenarios":"A delete/overwrite/RowDelta commit validates that no new data files matching its conflict-detection filter were added since its starting snapshot; a concurrent job appended rows matching the filter (e.g. same equality-delete keys or same row filter).","commonSituations":"Concurrent UPDATE/DELETE and INSERT on overlapping row ranges; equality-delete commits racing with appends of matching keys; stale compaction jobs validating old predicates.","solutions":["Refresh the table and rebuild/retry the operation against the current snapshot","Coordinate or serialize workloads touching overlapping predicates (partition the workload to disjoint filters)","Use a lower isolation/relax validation properties (e.g. write.delete.validation-pressure or skip delete validation) only when safe","Retry pattern: catch ValidationException, reload table, re-plan, re-commit"],"exampleFix":"// before\nrowDelta.commit(); // may throw ValidationException on concurrent appends\n// after\ntry {\n  rowDelta.commit();\n} catch (ValidationException e) {\n  Table refreshed = catalog.loadTable(id);\n  refreshed.newRowDelta()... // rebuild deletes against latest snapshot\n    .commit();\n}","handlingStrategy":"retry","validationCode":"// check for appends matching the filter since the planned snapshot\nSnapshot since = table.snapshot(plannedBaseSnapshotId);\nif (table.currentSnapshot().snapshotId() != plannedBaseSnapshotId) { /* re-validate or re-plan before commit */ }","typeGuard":null,"tryCatchPattern":"try { rowDelta.commit(); } catch (ValidationException e) { if (e.getMessage().startsWith(\"Found conflicting files that can contain records matching\")) { table.refresh(); /* rebuild deletes against latest snapshot */ } else throw e; }","preventionTips":["Minimize time between snapshot load and commit","Use retry-with-refresh loops for delete/overwrite commits in concurrent workloads","Design filters/partitions so concurrent writers operate on disjoint data ranges","Consider stronger coordination (external locks) for overlapping predicates"],"tags":["concurrency","conflict-detection","validation","delete-files"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}