apache/iceberg · error · UnsupportedOperationException

Cannot incrementally clean files when there are snapshots ou

Error message

Cannot incrementally clean files when there are snapshots outside of main

What it means

Even after expiration, the resulting table metadata may still contain snapshots outside the main ancestry (e.g. snapshots referenced by branches). Incremental cleanup cannot handle those, so RemoveSnapshots throws UnsupportedOperationException when hasNonMainSnapshots(current) is true.

Source

Thrown at core/src/main/java/org/apache/iceberg/RemoveSnapshots.java:425

            : new ReachableFileCleanup(
                ops.io(), deleteExecutorService, planExecutorService(), deleteFunc);

    cleanupStrategy.cleanFiles(base, current, cleanupLevel);
  }

  private void validateCleanupCanBeIncremental(TableMetadata current) {
    if (specifiedSnapshotId) {
      throw new UnsupportedOperationException(
          "Cannot clean files incrementally when snapshot IDs are specified");
    }

    if (hasRemovedNonMainAncestors(base, current)) {
      throw new UnsupportedOperationException(
          "Cannot incrementally clean files when snapshots outside of main ancestry were removed");
    }

    if (hasNonMainSnapshots(current)) {
      throw new UnsupportedOperationException(
          "Cannot incrementally clean files when there are snapshots outside of main");
    }
  }

  private boolean hasRemovedNonMainAncestors(
      TableMetadata beforeExpiration, TableMetadata afterExpiration) {
    Set<Long> mainAncestors = mainAncestors(beforeExpiration);
    for (Snapshot snapshotBeforeExpiration : beforeExpiration.snapshots()) {
      boolean removedSnapshot =
          afterExpiration.snapshot(snapshotBeforeExpiration.snapshotId()) == null;
      boolean snapshotInMainAncestry =
          mainAncestors.contains(snapshotBeforeExpiration.snapshotId());
      if (removedSnapshot && !snapshotInMainAncestry) {
        return true;
      }
    }

    return false;

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use full file cleanup (don't request incremental) when branches/non-main snapshots exist
2. Delete or merge the extra branches (removeBranch / removeSnapshots on those branches) so all snapshots are on main, then run incremental cleanup
3. Switch to timestamp-based expiration with full cleanup as a one-time consolidation step

Example fix

// before
expireSnapshots(table).withIncrementalCleanup().cleanExpiredFiles(true).execute();
// after
// remove extra branches first
table.manageSnapshots().dropBranch("feature-x").commit();
expireSnapshots(table).withIncrementalCleanup().cleanExpiredFiles(true).execute();

TableMetadata current = ((HasTableOperations) table).operations().current();
Set<Long> main = Sets.newHashSet(current.mainAncestorIds());
boolean nonMainPresent = current.snapshots().stream().anyMatch(s -> !main.contains(s.snapshotId()));
if (nonMainPresent) { /* drop branches or use full cleanup */ }

try {
  expireSnapshots(table).withIncrementalCleanup().execute();
} catch (UnsupportedOperationException e) {
  expireSnapshots(table).cleanExpiredFiles(true).execute(); // full cleanup
}

When it happens

Trigger: expireSnapshots with incremental cleanup on a table that retains non-main snapshots after expiration — typically snapshots still referenced by active branches or tags.

Common situations: Tables with long-lived feature branches; branch-based pipelines (WAP); users enabling incremental cleanup without realizing branches keep snapshots alive.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Cannot incrementally clean files when snapshots outside of m
• cleanExpiredMetadata is not supported
• Failed to close manifest list: %s
• Failed to read manifest file: %s
• Cannot clean files incrementally when snapshot IDs are speci

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/52334c36d62dc1f5. Report an issue: GitHub.