{"record":{"id":"58bfd148f56319be","repo":"apache/iceberg","slug":"cannot-incrementally-clean-files-when-snapshots-ou","errorCode":null,"errorMessage":"Cannot incrementally clean files when snapshots outside of main ancestry were removed","messagePattern":"Cannot incrementally clean files when snapshots outside of main ancestry were removed","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/RemoveSnapshots.java","lineNumber":420,"sourceCode":"\n    FileCleanupStrategy cleanupStrategy =\n        incrementalCleanup\n            ? new IncrementalFileCleanup(\n                ops.io(), deleteExecutorService, planExecutorService(), deleteFunc)\n            : new ReachableFileCleanup(\n                ops.io(), deleteExecutorService, planExecutorService(), deleteFunc);\n\n    cleanupStrategy.cleanFiles(base, current, cleanupLevel);\n  }\n\n  private void validateCleanupCanBeIncremental(TableMetadata current) {\n    if (specifiedSnapshotId) {\n      throw new UnsupportedOperationException(\n          \"Cannot clean files incrementally when snapshot IDs are specified\");\n    }\n\n    if (hasRemovedNonMainAncestors(base, current)) {\n      throw new UnsupportedOperationException(\n          \"Cannot incrementally clean files when snapshots outside of main ancestry were removed\");\n    }\n\n    if (hasNonMainSnapshots(current)) {\n      throw new UnsupportedOperationException(\n          \"Cannot incrementally clean files when there are snapshots outside of main\");\n    }\n  }\n\n  private boolean hasRemovedNonMainAncestors(\n      TableMetadata beforeExpiration, TableMetadata afterExpiration) {\n    Set<Long> mainAncestors = mainAncestors(beforeExpiration);\n    for (Snapshot snapshotBeforeExpiration : beforeExpiration.snapshots()) {\n      boolean removedSnapshot =\n          afterExpiration.snapshot(snapshotBeforeExpiration.snapshotId()) == null;\n      boolean snapshotInMainAncestry =\n          mainAncestors.contains(snapshotBeforeExpiration.snapshotId());\n      if (removedSnapshot && !snapshotInMainAncestry) {","sourceCodeStart":402,"sourceCodeEnd":438,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/RemoveSnapshots.java#L402-L438","documentation":"Incremental cleanup assumes expiration only removes snapshots from the main ancestry. If hasRemovedNonMainAncestors detects that snapshots outside the main ancestry were removed by this expiration, RemoveSnapshots throws UnsupportedOperationException because the incremental file-cleanup logic cannot safely determine deletable files in that case.","triggerScenarios":"expireSnapshots with incremental cleanup where the base metadata contained non-main-ancestor snapshots (e.g. orphaned snapshots from failed branches/writes) and this expiration removed them.","commonSituations":"Tables with snapshots from secondary branches or aborted writes; after CherryPick/Write-Audit-Publish style flows leaving non-main ancestors; migrating tables that contain legacy snapshot graphs.","solutions":["Use full (non-incremental) file cleanup so all reachable files across the whole metadata graph are computed safely","First expire with incremental cleanup only after non-main snapshots are already gone; otherwise fall back to full cleanup","Restructure branch/ancestry usage: retire non-main branches before switching to incremental cleanup"],"exampleFix":"// before\nexpireSnapshots(table).withIncrementalCleanup().cleanExpiredFiles(true).execute();\n// after\nexpireSnapshots(table).cleanExpiredFiles(true).execute(); // full cleanup handles removed non-main ancestors","handlingStrategy":"validation","validationCode":"// detect non-main ancestor snapshots before choosing incremental cleanup\nTableMetadata base = ((HasTableOperations) table).operations().current();\nboolean hasNonMain = base.snapshots().stream()\n    .anyMatch(s -> !base.mainAncestorIds().contains(s.snapshotId()));\nif (hasNonMain) { /* use full cleanup instead */ }","typeGuard":null,"tryCatchPattern":"try {\n  expireSnapshots(table).withIncrementalCleanup().execute();\n} catch (UnsupportedOperationException e) {\n  // fall back to full cleanup\n  expireSnapshots(table).cleanExpiredFiles(true).execute();\n}","preventionTips":["Prefer full cleanup on tables with branch history or WAP flows","Clean up secondary branches before adopting incremental cleanup","Audit snapshot ancestry before switching cleanup strategies"],"tags":["iceberg","snapshot-expiration","branches"],"backgroundTag":"conflicting-config-options","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}