{"record":{"id":"365573063665af33","repo":"apache/iceberg","slug":"cannot-commit-rewrite-because-of-a-validationexcep-365573","errorCode":null,"errorMessage":"Cannot commit rewrite because of a ValidationException or CommitFailedException. This usually means that this rewrite has conflicted with another concurrent Iceberg operation. To reduce the likelihood of conflicts, set %s which will break up the rewrite into multiple smaller commits controlled by %s. Separate smaller rewrite commits can succeed independently while any commits that conflict with another Iceberg operation will be ignored. This mode will create additional snapshots in the table history, one for each commit.","messagePattern":"Cannot commit rewrite because of a ValidationException or CommitFailedException\\. This usually means that this rewrite has conflicted with another concurrent Iceberg operation\\. To reduce the likelihood of conflicts, set %s which will break up the rewrite into multiple smaller commits controlled by %s\\. Separate smaller rewrite commits can succeed independently while any commits that conflict with another Iceberg operation will be ignored\\. This mode will create additional snapshots in the table history, one for each commit\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java","lineNumber":248,"sourceCode":"      Tasks.foreach(rewrittenGroups).suppressFailureWhenFinished().run(commitManager::abort);\n      throw e;\n    } finally {\n      rewriteService.shutdown();\n    }\n\n    try {\n      commitManager.commitOrClean(Sets.newHashSet(rewrittenGroups));\n    } catch (ValidationException | CommitFailedException e) {\n      String errorMessage =\n          String.format(\n              \"Cannot commit rewrite because of a ValidationException or CommitFailedException. This usually means that \"\n                  + \"this rewrite has conflicted with another concurrent Iceberg operation. To reduce the likelihood of \"\n                  + \"conflicts, set %s which will break up the rewrite into multiple smaller commits controlled by %s. \"\n                  + \"Separate smaller rewrite commits can succeed independently while any commits that conflict with \"\n                  + \"another Iceberg operation will be ignored. This mode will create additional snapshots in the table \"\n                  + \"history, one for each commit.\",\n              PARTIAL_PROGRESS_ENABLED, PARTIAL_PROGRESS_MAX_COMMITS);\n      throw new RuntimeException(errorMessage, e);\n    }\n\n    List<FileGroupRewriteResult> rewriteResults =\n        rewrittenGroups.stream()\n            .map(RewritePositionDeletesGroup::asResult)\n            .collect(Collectors.toList());\n\n    return ImmutableRewritePositionDeleteFiles.Result.builder()\n        .rewriteResults(rewriteResults)\n        .build();\n  }\n\n  private Result doExecuteWithPartialProgress(\n      FileRewritePlan<\n              FileGroupInfo, PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup>\n          plan,\n      RewritePositionDeletesCommitManager commitManager) {\n    ExecutorService rewriteService = rewriteService();","sourceCodeStart":230,"sourceCodeEnd":266,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java#L230-L266","documentation":"RewritePositionDeleteFilesSparkAction.doExecute wraps a ValidationException or CommitFailedException from the final table commit in a RuntimeException. The message explains that the rewrite conflicted with a concurrent Iceberg operation and suggests enabling partial progress mode (PARTIAL_PROGRESS_ENABLED / PARTIAL_PROGRESS_MAX_COMMITS) so smaller commits can succeed independently.","triggerScenarios":"Running rewritePositionDeletes() while another concurrent operation (compaction, expireSnapshots, writes, another rewrite) commits to the same table, causing the single final commit to fail validation or commit-retry exhaustion.","commonSituations":"Long-running rewrites on actively written tables; parallel maintenance jobs; timeouts when many rewrites compete for commits.","solutions":["Enable partial progress: set rewrite.position-delete.partial-progress.enabled=true and rewrite.partial-progress.max-commits to a positive value so groups commit independently.","Re-run the rewrite action; conflicting groups will be retried against the new table state.","Schedule the rewrite to avoid concurrent write/maintenance jobs on the same table."],"exampleFix":"// before\nsparkActions.rewritePositionDeletes(table).execute();\n// after\nsparkActions.rewritePositionDeletes(table)\n    .option(\"partial-progress.enabled\", \"true\")\n    .option(\"partial-progress.max-commits\", \"10\")\n    .execute();","handlingStrategy":"validation","validationCode":"if (!table.properties().containsKey(RewritePositionDeleteFilesSparkAction.PARTIAL_PROGRESS_ENABLED)) { /* consider enabling partial progress on busy tables */ }","typeGuard":null,"tryCatchPattern":"try { action.execute(); } catch (RuntimeException e) { if (e.getMessage() != null && e.getMessage().contains(\"Cannot commit rewrite\")) { /* retry or enable partial progress */ } else throw e; }","preventionTips":["Enable partial-progress.enabled for rewrites on tables with concurrent writers","Schedule rewrites outside heavy write windows","Retry the rewrite action; conflicting groups are re-planned against latest table state"],"tags":["spark","commit-conflict","rewrite-position-deletes","concurrency"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}