apache/iceberg · error

Failure during rewrite process for group {}

Error message

Failure during rewrite process for group {}

What it means

RewritePositionDeleteFilesSparkAction logs this warning via Tasks.foreach onFailure when rewriting a group of position delete files throws. With stopOnFailure and noRetry, the first failure aborts the parallel loop and the exception propagates, ending execute() with an error while identifying which group failed.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java:210

  private Result doExecute(
      FileRewritePlan<
              FileGroupInfo, PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup>
          plan,
      RewritePositionDeletesCommitManager commitManager) {
    ExecutorService rewriteService = rewriteService();

    ConcurrentLinkedQueue<RewritePositionDeletesGroup> rewrittenGroups =
        new ConcurrentLinkedQueue<>();

    Tasks.Builder<RewritePositionDeletesGroup> rewriteTaskBuilder =
        Tasks.foreach(plan.groups())
            .executeWith(rewriteService)
            .stopOnFailure()
            .noRetry()
            .onFailure(
                (fileGroup, exception) ->
                    LOG.warn(
                        "Failure during rewrite process for group {}",
                        fileGroup.info(),
                        exception));

    try {
      rewriteTaskBuilder.run(fileGroup -> rewrittenGroups.add(rewriteDeleteFiles(plan, fileGroup)));
    } catch (Exception e) {
      // At least one rewrite group failed, clean up all completed rewrites
      LOG.error(
          "Cannot complete rewrite, {} is not enabled and one of the file set groups failed to "
              + "be rewritten. This error occurred during the writing of new files, not during the commit process. This "
              + "indicates something is wrong that doesn't involve conflicts with other Iceberg operations. Enabling "
              + "{} may help in this case but the root cause should be investigated. Cleaning up {} groups which finished "
              + "being written.",
          PARTIAL_PROGRESS_ENABLED,
          PARTIAL_PROGRESS_ENABLED,
          rewrittenGroups.size(),
          e);

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Read the chained exception to see the root cause and the failing group's info
2. Re-run the action after resolving the underlying IO/validation failure
3. Split work into smaller groups (fewer files per group) to isolate the failing file
4. Check for concurrent writers that may have removed delete files mid-rewrite

table.refresh();
new ValidateFromSnapshotValidation().validate(...); // or simply re-plan right before execute to minimize staleness

try {
  action.execute();
} catch (Exception e) {
  LOG.error("Rewrite of position delete files failed for group; cause: {}", e.getCause(), e);
  // inspect group info in logs, fix IO/concurrency issue, then retry
}

When it happens

Trigger: Calling RewritePositionDeleteFilesSparkAction.execute() where rewrite of a file group fails during doExecute — e.g. delete-file reads fail, commit validation fails, or a Spark job error occurs for that group.

Common situations: Corrupt or missing position delete files; concurrent table commits invalidating the plan; Spark task failures on large delete-file groups; IO errors on object storage.

Related errors

• Failure during rewrite process for group {}
• Cannot commit rewrite because of a ValidationException or Co
• Failure during rewrite process for group {}
• Failure during rewrite process for group {}
• Cannot commit rewrite because of a ValidationException or Co

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/0ce5ceb774b03cb0. Report an issue: GitHub.