{"record":{"id":"18f5390041260426","repo":"apache/iceberg","slug":"failure-during-rewrite-process-for-group-18f539","errorCode":null,"errorMessage":"Failure during rewrite process for group {}","messagePattern":"Failure during rewrite process for group (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteDataFilesSparkAction.java","lineNumber":280,"sourceCode":"    return new RewriteDataFilesCommitManager(\n        table, startingSnapshotId, useStartingSequenceNumber, commitSummary(), branch);\n  }\n\n  private Builder doExecute(\n      FileRewritePlan<FileGroupInfo, FileScanTask, DataFile, RewriteFileGroup> plan,\n      RewriteDataFilesCommitManager commitManager) {\n    ExecutorService rewriteService = rewriteService();\n\n    ConcurrentLinkedQueue<RewriteFileGroup> rewrittenGroups = new ConcurrentLinkedQueue<>();\n\n    Tasks.Builder<RewriteFileGroup> rewriteTaskBuilder =\n        Tasks.foreach(plan.groups())\n            .executeWith(rewriteService)\n            .stopOnFailure()\n            .noRetry()\n            .onFailure(\n                (fileGroup, exception) -> {\n                  LOG.warn(\n                      \"Failure during rewrite process for group {}\", fileGroup.info(), exception);\n                });\n\n    try {\n      rewriteTaskBuilder.run(\n          fileGroup -> {\n            rewrittenGroups.add(rewriteFiles(plan, fileGroup));\n          });\n    } catch (Exception e) {\n      // At least one rewrite group failed, clean up all completed rewrites\n      LOG.error(\n          \"Cannot complete rewrite, {} is not enabled and one of the file set groups failed to \"\n              + \"be rewritten. This error occurred during the writing of new files, not during the commit process. This \"\n              + \"indicates something is wrong that doesn't involve conflicts with other Iceberg operations. Enabling \"\n              + \"{} may help in this case but the root cause should be investigated. Cleaning up {} groups which finished \"\n              + \"being written.\",\n          PARTIAL_PROGRESS_ENABLED,\n          PARTIAL_PROGRESS_ENABLED,","sourceCodeStart":262,"sourceCodeEnd":298,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteDataFilesSparkAction.java#L262-L298","documentation":"A logged warning emitted by RewriteDataFilesSparkAction when rewriting a file group fails. Tasks.foreach over plan.groups() with stopOnFailure/noRetry records the per-group exception in the onFailure handler; the group's partial result is discarded and remaining groups continue, so the compaction run degrades instead of failing wholesale.","triggerScenarios":"Running optimize/rewriteDataFiles when one group's rewrite job throws — schema/eviction errors reading files, executor OOMs on large groups, corrupted data files, or write failures to the target location (permissions, quota).","commonSituations":"Corrupt legacy Parquet/Avro files from older writers; max-file-group-size too large causing executor OOM; target bucket/permission misconfigurations; incompatible codec or encryption settings; transient S3 throttling during writes.","solutions":["Read the chained exception for the specific group; fix the root cause (corrupt file, permissions, memory).","Reduce max-file-group-size / max-files-per-group to avoid executor OOM on large groups.","Re-run the rewrite; successfully completed groups are skipped, only failed groups are retried.","Skip or repair known-corrupt files before compaction (validate with Spark reads).","Increase executor memory if failures are OOM-driven: spark.executor.memory or memoryOverhead."],"exampleFix":"// before\nspark.table(\"db.tbl\").call(\"rewrite_data_files\");  // group fails on OOM\n// after\nspark.table(\"db.tbl\").call(\"rewrite_data_files\",\n  Map.of(\"max-file-group-size-bytes\", \"1073741824\"));  // 1GB groups\nspark.conf.set(\"spark.executor.memory\", \"8g\");","handlingStrategy":"retry","validationCode":"// validate data files are readable before compaction\nspark.read.format(\"iceberg\").load(\"db.tbl\").limit(1000).collectAsList();","typeGuard":null,"tryCatchPattern":"try { rewrite.execute(); } catch (Exception e) { /* group failures logged; fix cause and re-run */ }","preventionTips":["Repair or quarantine corrupt files before rewriting","Right-size max-file-group-size to avoid executor OOM","Increase executor memory/overhead for large groups","Fix storage permissions/quota on the target location"],"tags":["spark","compaction","rewrite","data-files"],"backgroundTag":"internal-invariant-violation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}