{"record":{"id":"5d8f9980f796451b","repo":"apache/iceberg","slug":"cannot-commit-file-that-conflicts-with-existing-pa","errorCode":null,"errorMessage":"Cannot commit file that conflicts with existing partition: %s","messagePattern":"Cannot commit file that conflicts with existing partition: (.+?)","errorType":"validation","errorClass":"ValidationException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/BaseReplacePartitions.java","lineNumber":122,"sourceCode":"            currentMetadata, startingSnapshotId, Expressions.alwaysTrue(), parent);\n      } else {\n        validateDeletedDataFiles(currentMetadata, startingSnapshotId, replacedPartitions, parent);\n        validateNoNewDeleteFiles(currentMetadata, startingSnapshotId, replacedPartitions, parent);\n      }\n    }\n  }\n\n  @Override\n  public List<ManifestFile> apply(TableMetadata base, Snapshot snapshot) {\n    if (dataSpec().isUnpartitioned()) {\n      // replace all data in an unpartitioned table\n      deleteByRowFilter(Expressions.alwaysTrue());\n    }\n\n    try {\n      return super.apply(base, snapshot);\n    } catch (ManifestFilterManager.DeleteException e) {\n      throw new ValidationException(\n          \"Cannot commit file that conflicts with existing partition: %s\", e.partition());\n    }\n  }\n}\n","sourceCodeStart":104,"sourceCodeEnd":127,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/BaseReplacePartitions.java#L104-L127","documentation":"ReplacePartitions overwrites entire partitions by deleting all rows of the incoming files' partitions. When super.apply() detects (via ManifestFilterManager.DeleteException) that the commit would delete a partition that contains files not covered by the replace — e.g. a concurrent commit added files to that partition — it throws this ValidationException to preserve atomicity.","triggerScenarios":"Committing a ReplacePartitions operation whose data files touch partitions that contain other data files not being replaced (typical with non-identity/dynamic partition layouts or a concurrent writer that added files to the same partition between plan and commit).","commonSituations":"Two jobs concurrently overwriting the same partition; overwriting a partition where files were written with a different spec; replace-partitions against tables where rows of one partition span multiple writers.","solutions":["Re-run the job: re-plan the ReplacePartitions against the latest snapshot so the conflicting files are covered by the delete filter.","Use a retry loop on ValidationException/CommitFailedException around the overwrite.","Switch to overwrite(newOverwrite()) with explicit file-level overwrites when only some files should be replaced.","Ensure application-level locking so only one writer replaces a given partition at a time."],"exampleFix":"// before\ntable.newReplacePartitions().appendFile(file).commit(); // throws on conflict\n// after\nTasks.foreach(files)\n    .retry(3)\n    .exponentialBackoff(100, 3000)\n    .run(f -> table.refresh().newReplacePartitions().appendFile(f).commit());","handlingStrategy":"retry","validationCode":"// before committing, confirm the target partitions have no un-replaced files\nList<DataFile> partitionFiles = Files.tableFiles(table)\n    .filter(f -> targetSpec.partitionType().equals(f.partitionSpec(specId).partitionType()))\n    .collect(Collectors.toList());\n// ensure all partitionFiles are being overwritten by the ReplacePartitions","typeGuard":null,"tryCatchPattern":"try {\n  table.newReplacePartitions().appendFiles(files).commit();\n} catch (ValidationException e) {\n  if (e.getMessage().startsWith(\"Cannot commit file that conflicts with existing partition\")) {\n    table.refresh(); // re-plan against latest snapshot and retry\n  } else throw e;\n}","preventionTips":["Retry ReplacePartitions with a refreshed table on conflict","Serialize partition overwrites with application-level locks","Avoid overwriting partitions while other writers append to them","Prefer OverwriteFiles for file-level (not whole-partition) replacement"],"tags":["validation","commit-conflict","replace-partitions","concurrency"],"backgroundTag":"conflicting-config-options","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}