{"record":{"id":"d5028a0ad5b00c41","repo":"apache/iceberg","slug":"cannot-commit-rewrite-because-of-a-validationexcep-d5028a","errorCode":null,"errorMessage":"Cannot commit rewrite because of a ValidationException or CommitFailedException. This usually means that this rewrite has conflicted with another concurrent Iceberg operation. To reduce the likelihood of conflicts, set %s which will break up the rewrite into multiple smaller commits controlled by %s. Separate smaller rewrite commits can succeed independently while any commits that conflict with another Iceberg operation will be ignored. This mode will create additional snapshots in the table history, one for each commit.","messagePattern":"Cannot commit rewrite because of a ValidationException or CommitFailedException\\. This usually means that this rewrite has conflicted with another concurrent Iceberg operation\\. To reduce the likelihood of conflicts, set %s which will break up the rewrite into multiple smaller commits controlled by %s\\. Separate smaller rewrite commits can succeed independently while any commits that conflict with another Iceberg operation will be ignored\\. This mode will create additional snapshots in the table history, one for each commit\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java","lineNumber":248,"sourceCode":"      Tasks.foreach(rewrittenGroups).suppressFailureWhenFinished().run(commitManager::abort);\n      throw e;\n    } finally {\n      rewriteService.shutdown();\n    }\n\n    try {\n      commitManager.commitOrClean(Sets.newHashSet(rewrittenGroups));\n    } catch (ValidationException | CommitFailedException e) {\n      String errorMessage =\n          String.format(\n              \"Cannot commit rewrite because of a ValidationException or CommitFailedException. This usually means that \"\n                  + \"this rewrite has conflicted with another concurrent Iceberg operation. To reduce the likelihood of \"\n                  + \"conflicts, set %s which will break up the rewrite into multiple smaller commits controlled by %s. \"\n                  + \"Separate smaller rewrite commits can succeed independently while any commits that conflict with \"\n                  + \"another Iceberg operation will be ignored. This mode will create additional snapshots in the table \"\n                  + \"history, one for each commit.\",\n              PARTIAL_PROGRESS_ENABLED, PARTIAL_PROGRESS_MAX_COMMITS);\n      throw new RuntimeException(errorMessage, e);\n    }\n\n    List<FileGroupRewriteResult> rewriteResults =\n        rewrittenGroups.stream()\n            .map(RewritePositionDeletesGroup::asResult)\n            .collect(Collectors.toList());\n\n    return ImmutableRewritePositionDeleteFiles.Result.builder()\n        .rewriteResults(rewriteResults)\n        .build();\n  }\n\n  private Result doExecuteWithPartialProgress(\n      FileRewritePlan<\n              FileGroupInfo, PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup>\n          plan,\n      RewritePositionDeletesCommitManager commitManager) {\n    ExecutorService rewriteService = rewriteService();","sourceCodeStart":230,"sourceCodeEnd":266,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java#L230-L266","documentation":"RewritePositionDeleteFiles commits all rewrite groups in one transaction unless partial progress is enabled. A ValidationException or CommitFailedException escaping the commit means the rewrite conflicted with another concurrent table operation; the action wraps it in a RuntimeException with guidance to enable partial-progress commits.","triggerScenarios":"doExecute (invoked via execute) commits rewritten position-delete groups while another concurrent Iceberg operation changed the table, producing a ValidationException (requirement conflict) or exhausted CommitFailedException retries.","commonSituations":"Concurrent compaction/writes to the same table while deleting rewrite is running; long-running rewrite whose planned requirements were invalidated; repeated commit retries failing under contention.","solutions":["Enable partial-progress.enabled so rewrites are committed in smaller independent groups","Tune partial-progress.max-commits to control how the rewrite is split into commits","Serialize conflicting jobs — avoid running delete-file rewrite concurrently with other rewrites/expiry, then re-run"],"exampleFix":"// before\nactions.rewritePositionDeletes(table).execute();\n// after\nactions.rewritePositionDeletes(table)\n    .option(\"partial-progress.enabled\", \"true\")\n    .option(\"partial-progress.max-commits\", \"10\")\n    .execute();","handlingStrategy":"try-catch","validationCode":"// ensure no concurrent table mutations\nboolean contended = concurrentJobsActive(table);\nif (contended) { enablePartialProgress(); }","typeGuard":null,"tryCatchPattern":"try { action.execute(); } catch (RuntimeException e) { if (e.getCause() instanceof ValidationException || e.getCause() instanceof CommitFailedException) { /* retry with partial-progress.enabled */ } }","preventionTips":["Avoid running delete-file rewrites concurrently with other rewrites or expireSnapshots","Enable partial-progress for tables with frequent writes","Run maintenance in serialized maintenance windows"],"tags":["spark","iceberg","commit-conflict","position-deletes"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}