apache/iceberg · error · UnsupportedOperationException

Cannot incrementally clean files when snapshots outside of m

Error message

Cannot incrementally clean files when snapshots outside of main ancestry were removed

What it means

Incremental cleanup assumes expiration only removes snapshots from the main ancestry. If hasRemovedNonMainAncestors detects that snapshots outside the main ancestry were removed by this expiration, RemoveSnapshots throws UnsupportedOperationException because the incremental file-cleanup logic cannot safely determine deletable files in that case.

Source

Thrown at core/src/main/java/org/apache/iceberg/RemoveSnapshots.java:420

    FileCleanupStrategy cleanupStrategy =
        incrementalCleanup
            ? new IncrementalFileCleanup(
                ops.io(), deleteExecutorService, planExecutorService(), deleteFunc)
            : new ReachableFileCleanup(
                ops.io(), deleteExecutorService, planExecutorService(), deleteFunc);

    cleanupStrategy.cleanFiles(base, current, cleanupLevel);
  }

  private void validateCleanupCanBeIncremental(TableMetadata current) {
    if (specifiedSnapshotId) {
      throw new UnsupportedOperationException(
          "Cannot clean files incrementally when snapshot IDs are specified");
    }

    if (hasRemovedNonMainAncestors(base, current)) {
      throw new UnsupportedOperationException(
          "Cannot incrementally clean files when snapshots outside of main ancestry were removed");
    }

    if (hasNonMainSnapshots(current)) {
      throw new UnsupportedOperationException(
          "Cannot incrementally clean files when there are snapshots outside of main");
    }
  }

  private boolean hasRemovedNonMainAncestors(
      TableMetadata beforeExpiration, TableMetadata afterExpiration) {
    Set<Long> mainAncestors = mainAncestors(beforeExpiration);
    for (Snapshot snapshotBeforeExpiration : beforeExpiration.snapshots()) {
      boolean removedSnapshot =
          afterExpiration.snapshot(snapshotBeforeExpiration.snapshotId()) == null;
      boolean snapshotInMainAncestry =
          mainAncestors.contains(snapshotBeforeExpiration.snapshotId());
      if (removedSnapshot && !snapshotInMainAncestry) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use full (non-incremental) file cleanup so all reachable files across the whole metadata graph are computed safely
2. First expire with incremental cleanup only after non-main snapshots are already gone; otherwise fall back to full cleanup
3. Restructure branch/ancestry usage: retire non-main branches before switching to incremental cleanup

Example fix

// before
expireSnapshots(table).withIncrementalCleanup().cleanExpiredFiles(true).execute();
// after
expireSnapshots(table).cleanExpiredFiles(true).execute(); // full cleanup handles removed non-main ancestors

// detect non-main ancestor snapshots before choosing incremental cleanup
TableMetadata base = ((HasTableOperations) table).operations().current();
boolean hasNonMain = base.snapshots().stream()
    .anyMatch(s -> !base.mainAncestorIds().contains(s.snapshotId()));
if (hasNonMain) { /* use full cleanup instead */ }

try {
  expireSnapshots(table).withIncrementalCleanup().execute();
} catch (UnsupportedOperationException e) {
  // fall back to full cleanup
  expireSnapshots(table).cleanExpiredFiles(true).execute();
}

When it happens

Trigger: expireSnapshots with incremental cleanup where the base metadata contained non-main-ancestor snapshots (e.g. orphaned snapshots from failed branches/writes) and this expiration removed them.

Common situations: Tables with snapshots from secondary branches or aborted writes; after CherryPick/Write-Audit-Publish style flows leaving non-main ancestors; migrating tables that contain legacy snapshot graphs.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Cannot incrementally clean files when there are snapshots ou
• cleanExpiredMetadata is not supported
• Failed to close manifest list: %s
• Failed to read manifest file: %s
• Cannot clean files incrementally when snapshot IDs are speci

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/58bfd148f56319be. Report an issue: GitHub.