{"record":{"id":"0ce5ceb774b03cb0","repo":"apache/iceberg","slug":"failure-during-rewrite-process-for-group-0ce5ce","errorCode":null,"errorMessage":"Failure during rewrite process for group {}","messagePattern":"Failure during rewrite process for group (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java","lineNumber":210,"sourceCode":"\n  private Result doExecute(\n      FileRewritePlan<\n              FileGroupInfo, PositionDeletesScanTask, DeleteFile, RewritePositionDeletesGroup>\n          plan,\n      RewritePositionDeletesCommitManager commitManager) {\n    ExecutorService rewriteService = rewriteService();\n\n    ConcurrentLinkedQueue<RewritePositionDeletesGroup> rewrittenGroups =\n        new ConcurrentLinkedQueue<>();\n\n    Tasks.Builder<RewritePositionDeletesGroup> rewriteTaskBuilder =\n        Tasks.foreach(plan.groups())\n            .executeWith(rewriteService)\n            .stopOnFailure()\n            .noRetry()\n            .onFailure(\n                (fileGroup, exception) ->\n                    LOG.warn(\n                        \"Failure during rewrite process for group {}\",\n                        fileGroup.info(),\n                        exception));\n\n    try {\n      rewriteTaskBuilder.run(fileGroup -> rewrittenGroups.add(rewriteDeleteFiles(plan, fileGroup)));\n    } catch (Exception e) {\n      // At least one rewrite group failed, clean up all completed rewrites\n      LOG.error(\n          \"Cannot complete rewrite, {} is not enabled and one of the file set groups failed to \"\n              + \"be rewritten. This error occurred during the writing of new files, not during the commit process. This \"\n              + \"indicates something is wrong that doesn't involve conflicts with other Iceberg operations. Enabling \"\n              + \"{} may help in this case but the root cause should be investigated. Cleaning up {} groups which finished \"\n              + \"being written.\",\n          PARTIAL_PROGRESS_ENABLED,\n          PARTIAL_PROGRESS_ENABLED,\n          rewrittenGroups.size(),\n          e);","sourceCodeStart":192,"sourceCodeEnd":228,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java#L192-L228","documentation":"RewritePositionDeleteFilesSparkAction logs this warning via Tasks.foreach onFailure when rewriting a group of position delete files throws. With stopOnFailure and noRetry, the first failure aborts the parallel loop and the exception propagates, ending execute() with an error while identifying which group failed.","triggerScenarios":"Calling RewritePositionDeleteFilesSparkAction.execute() where rewrite of a file group fails during doExecute — e.g. delete-file reads fail, commit validation fails, or a Spark job error occurs for that group.","commonSituations":"Corrupt or missing position delete files; concurrent table commits invalidating the plan; Spark task failures on large delete-file groups; IO errors on object storage.","solutions":["Read the chained exception to see the root cause and the failing group's info","Re-run the action after resolving the underlying IO/validation failure","Split work into smaller groups (fewer files per group) to isolate the failing file","Check for concurrent writers that may have removed delete files mid-rewrite"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"table.refresh();\nnew ValidateFromSnapshotValidation().validate(...); // or simply re-plan right before execute to minimize staleness","typeGuard":null,"tryCatchPattern":"try {\n  action.execute();\n} catch (Exception e) {\n  LOG.error(\"Rewrite of position delete files failed for group; cause: {}\", e.getCause(), e);\n  // inspect group info in logs, fix IO/concurrency issue, then retry\n}","preventionTips":["Minimize time between planning and commit to avoid concurrent-writer conflicts","Verify delete files are readable before rewriting","Run rewrites on tables with quiescent writers","Retry with smaller groups to isolate failing files"],"tags":["spark","position-deletes","rewrite","task-failure"],"backgroundTag":"job-execution-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}