apache/iceberg · error · UnsupportedOperationException

Found operation, cannot support incremental data in…

Error message

Found %s operation, cannot support incremental data in snapshots (%s, %s]

What it means

Incremental scans currently support only APPEND snapshots: any overwrite rewrites existing rows, which a difference-of-manifests incremental read cannot represent safely. snapshotsWithin walks the ancestor chain between the range endpoints and throws UnsupportedOperationException if it encounters an OVERWRITE snapshot, reporting the range.

Solutions

Choose a fromSnapshotId after the last overwrite snapshot so the range contains only appends.
Use the changelog scan (IncrementalChangelogScan) or table.scan() with row-level filters if you must read a range containing overwrites.
Audit snapshot operations in the range via SnapshotUtil.ancestorsBetween to find the offending overwrite snapshot.
Rewrite the workflow that produced the overwrite to use append-only semantics (e.g. copy-on-write delete+append) if incremental append scans are a hard requirement.

Example fix

// before
TableScan scan = table.incrementalScan(earliestId, currentId); // may include overwrites
// after
long safeFrom = SnapshotUtil.ancestorsBetween(currentId, null, table::snapshot).stream()
    .filter(id -> DataOperations.APPEND.equals(table.snapshot(id).operation()))
    .filter(id -> id > earliestId)
    .findFirst().orElse(earliestId);
TableScan scan = table.incrementalScan(safeFrom, currentId); // appends only

Defensive patterns

Strategy: try-catch

Validate before calling

import org.apache.iceberg.SnapshotUtil;
import org.apache.iceberg.DataOperations;
boolean rangeAppendOnly(long fromId, long toId, Table table) {
  return SnapshotUtil.ancestorsBetween(toId, fromId, table::snapshot).stream()
      .allMatch(id -> DataOperations.APPEND.equals(table.snapshot(id).operation()));
}
// check before calling incrementalScan; otherwise use changelog or a full scan

Try / catch

try {
  TableScan scan = table.incrementalScan(fromId, toId);
  scan.planFiles();
} catch (UnsupportedOperationException e) {
  // range contains an overwrite; fall back to changelog or full scan
  TableScan scan = table.incrementalChangelogScan().fromSnapshotId(fromId);
}

Prevention

Check snapshot operations in a range before incremental scans; keep maintenance overwrites outside scanned ranges.
Prefer changelog scans when row-level operations matter.
Snapshot audits with SnapshotUtil.ancestorsBetween become part of pipeline validation.

When it happens

Trigger: Calling table.incrementalScan(fromId, toId) (or .planFiles()/tasks()) where any ancestor snapshot between fromId (exclusive) and toId (inclusive) has operation 'overwrite'.

Common situations: Tables that received overwrite/compaction-with-overwrite (e.g. MERGE OVERWRITE, dynamic partition overwrite via a non-appending writer) inside the requested range; users scanning ranges that span maintenance jobs.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/265d1b21f76b9cb6. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/IncrementalDataTableScan.java:139

    return manifestGroup.planFiles();
  }

  @Override
  @SuppressWarnings("checkstyle:HiddenField")
  protected TableScan newRefinedScan(Table table, Schema schema, TableScanContext context) {
    return new IncrementalDataTableScan(table, schema, context);
  }

  private static List<Snapshot> snapshotsWithin(
      Table table, long fromSnapshotId, long toSnapshotId) {
    List<Snapshot> snapshots = Lists.newArrayList();
    for (Snapshot snapshot :
        SnapshotUtil.ancestorsBetween(toSnapshotId, fromSnapshotId, table::snapshot)) {
      // for now, incremental scan supports only appends
      if (snapshot.operation().equals(DataOperations.APPEND)) {
        snapshots.add(snapshot);
      } else if (snapshot.operation().equals(DataOperations.OVERWRITE)) {
        throw new UnsupportedOperationException(
            String.format(
                "Found %s operation, cannot support incremental data in snapshots (%s, %s]",
                DataOperations.OVERWRITE, fromSnapshotId, toSnapshotId));
      }
    }
    return snapshots;
  }

  private void validateSnapshotIdsRefinement(long newFromSnapshotId, long newToSnapshotId) {
    Set<Long> snapshotIdsRange =
        Sets.newHashSet(
            SnapshotUtil.ancestorIdsBetween(
                context().toSnapshotId(), context().fromSnapshotId(), table()::snapshot));
    // since snapshotIdsBetween return ids in range (fromSnapshotId, toSnapshotId]
    snapshotIdsRange.add(context().fromSnapshotId());
    Preconditions.checkArgument(
        snapshotIdsRange.contains(newFromSnapshotId),
        "from snapshot id %s not in existing snapshot ids range (%s, %s]",

View on GitHub (pinned to 86d9c8fc54)