apache/iceberg · error · ValidationException

Found conflicting deleted files that can apply to records ma

Error message

Found conflicting deleted files that can apply to records matching %s: %s

What it means

ValidationException thrown at commit when a concurrent snapshot deleted data files whose partitions fall within the provided PartitionSet, meaning those deletes can apply to records the current operation touches. Iceberg rejects the commit to preserve correctness under concurrent modifications. Like other validation failures, rebasing on the latest snapshot is required.

Source

Thrown at core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java:716

  }

  /**
   * Validates that no files matching a filter have been deleted from the table since a starting
   * snapshot.
   *
   * @param base table metadata to validate
   * @param startingSnapshotId id of the snapshot current at the start of the operation
   * @param partitionSet a partition set used to find deleted data files
   * @param parent ending snapshot on the branch being validated
   */
  protected void validateDeletedDataFiles(
      TableMetadata base, Long startingSnapshotId, PartitionSet partitionSet, Snapshot parent) {
    CloseableIterable<ManifestEntry<DataFile>> conflictEntries =
        deletedDataFiles(base, startingSnapshotId, null, partitionSet, parent);

    try (CloseableIterator<ManifestEntry<DataFile>> conflicts = conflictEntries.iterator()) {
      if (conflicts.hasNext()) {
        throw new ValidationException(
            "Found conflicting deleted files that can apply to records matching %s: %s",
            partitionSet,
            Iterators.toString(
                Iterators.transform(conflicts, entry -> entry.file().location().toString())));
      }

    } catch (IOException e) {
      throw new UncheckedIOException(
          String.format("Failed to validate no appends matching %s", partitionSet), e);
    }
  }

  /**
   * Returns an iterable of files matching a filter have been added to the table since a starting
   * snapshot.
   *
   * @param base table metadata to validate
   * @param startingSnapshotId id of the snapshot current at the start of the operation

View on GitHub (pinned to 86d9c8fc54)

Solutions

Refresh the table and rebase the operation on the current snapshot, then recompute the PartitionSet and retry.
Sequence conflicting jobs (compaction vs delete/merge) so they don't overlap on the same partitions.
Restrict the operation to partitions not concurrently being compacted/rewritten.
Use branch-based isolation (WAP branches) to stage and fast-forward atomically.

Example fix

// before
rowDelta.validateNoDeletedFiles(startSnapshotId, conflictingPartitions).commit(); // throws on concurrent compaction
// after
table.refresh();
RowDelta op = table.newRowDelta();
op.validateNoDeletedFiles(table.currentSnapshot().snapshotId(), conflictingPartitions);
op.commit(); // retry after rebase

Defensive patterns

Strategy: retry

Validate before calling

table.refresh(); // recompute partition set against current state
PartitionSet set = PartitionSet.create(table.specs());

Try / catch

try {
  operation.commit();
} catch (ValidationException e) {
  table.refresh();
  // rebase operation and recompute the PartitionSet, then retry
}

Prevention

Sequence compaction and DML on the same partitions
Refresh before validating and committing
Narrow operations to partitions not under concurrent rewrite

When it happens

Trigger: Calling validateNoDeletedFiles with a PartitionSet (e.g., from BaseOverwriteFiles.validate or RowDelta operations) while concurrent commits deleted data files in overlapping partitions since the starting snapshot.

Common situations: Concurrent partition-level rewrites (compaction) racing with row-level deletes/overwrites; Spark MERGE/DELETE jobs overlapping with rewriteDataFiles procedures; multi-writer pipelines without coordination.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/da9ad3248acad27e. Report an issue: GitHub.