{"record":{"id":"ec9384e20ade0e45","repo":"apache/iceberg","slug":"failed-to-commit-rewrite-cleaning-up-rewritten-fi","errorCode":null,"errorMessage":"Failed to commit rewrite, cleaning up rewritten files","messagePattern":"Failed to commit rewrite, cleaning up rewritten files","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"core/src/main/java/org/apache/iceberg/actions/BaseRewriteDataFilesAction.java","lineNumber":314,"sourceCode":"          });\n    } catch (IOException e) {\n      LOG.warn(\"Failed to close task iterator\", e);\n    }\n    return tasksGroupedByPartition.asMap();\n  }\n\n  private void replaceDataFiles(\n      Iterable<DataFile> deletedDataFiles,\n      Iterable<DataFile> addedDataFiles,\n      long startingSnapshotId) {\n    try {\n      doReplace(deletedDataFiles, addedDataFiles, startingSnapshotId);\n    } catch (CommitStateUnknownException e) {\n      LOG.warn(\"Commit state unknown, cannot clean up files that may have been committed\", e);\n      throw e;\n    } catch (Exception e) {\n      if (e instanceof CleanableFailure) {\n        LOG.warn(\"Failed to commit rewrite, cleaning up rewritten files\", e);\n        Tasks.foreach(Iterables.transform(addedDataFiles, ContentFile::location))\n            .noRetry()\n            .suppressFailureWhenFinished()\n            .onFailure((location, exc) -> LOG.warn(\"Failed to delete: {}\", location, exc))\n            .run(fileIO::deleteFile);\n      }\n\n      throw e;\n    }\n  }\n\n  @VisibleForTesting\n  void doReplace(\n      Iterable<DataFile> deletedDataFiles,\n      Iterable<DataFile> addedDataFiles,\n      long startingSnapshotId) {\n    RewriteFiles rewriteFiles = table.newRewrite().validateFromSnapshot(startingSnapshotId);\n","sourceCodeStart":296,"sourceCodeEnd":332,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/actions/BaseRewriteDataFilesAction.java#L296-L332","documentation":"If a rewrite commit in BaseRewriteDataFilesAction.replaceDataFiles() fails with an exception implementing CleanableFailure (i.e. the commit definitively failed and the new files were not committed), the action logs this warning and deletes the newly written data files to avoid orphans. This is the safe-cleanup branch, distinct from the CommitStateUnknown case where files are kept.","triggerScenarios":"doReplace() commit throws an exception implementing CleanableFailure during replaceDataFiles(); the action responds by deleting all addedDataFiles via FileIO with no retry.","commonSituations":"Validation failures at commit (concurrent table changes conflicting with the rewrite); optimistic-concurrency conflicts resolved as definite failures; deterministic commit rejections by the catalog.","solutions":["Inspect the logged CleanableFailure cause to understand why the commit definitively failed.","Re-run the RewriteDataFiles action after resolving the conflict (e.g. fewer concurrent writers, retry).","If conflicts are frequent, reduce file-group overlap or commit concurrently with other jobs less aggressively.","Check that the cleanup deletion itself succeeded — per-file delete failures are logged as 'Failed to delete' warnings; clean leftovers with RemoveOrphanFiles."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  replace(deleted, added, snapshotId);\n} catch (CleanableFailure e) {\n  added.forEach(file -> fileIO.deleteFile(file.location())); // definite failure: safe to clean up\n  throw e;\n} catch (CommitStateUnknownException e) {\n  // do NOT delete added files; state unknown\n  throw e;\n}","preventionTips":["Implement CleanableFailure for definitive commit failures so cleanup can proceed","Schedule RemoveOrphanFiles to catch any delete failures during cleanup","Limit concurrent writers to reduce commit conflicts","Re-run rewrites after resolving the reported failure cause"],"tags":["commit","cleanup","rewrite","concurrency"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}