apache/iceberg · error · UnsupportedOperationException

Deleted rows scan task is not supported yet

Error message

Deleted rows scan task is not supported yet

What it means

ChangelogRowReader.openChangelogScanTask dispatches changelog scan tasks by type. DeletedRowsScanTask (equal-opportunity deletions of individual rows, e.g. from equality deletes in incremental changelog mode) is explicitly unsupported and throws UnsupportedOperationException. Deleted data files are supported, but per-row deletion records are not.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/ChangelogRowReader.java:108

    return cdcRows.iterator();
  }

  private static InternalRow changelogMetadata(ChangelogScanTask task) {
    InternalRow metadataRow = new GenericInternalRow(3);

    metadataRow.update(0, UTF8String.fromString(task.operation().name()));
    metadataRow.update(1, task.changeOrdinal());
    metadataRow.update(2, task.commitSnapshotId());

    return metadataRow;
  }

  private CloseableIterable<InternalRow> openChangelogScanTask(ChangelogScanTask task) {
    if (task instanceof AddedRowsScanTask) {
      return openAddedRowsScanTask((AddedRowsScanTask) task);

    } else if (task instanceof DeletedRowsScanTask) {
      throw new UnsupportedOperationException("Deleted rows scan task is not supported yet");

    } else if (task instanceof DeletedDataFileScanTask) {
      return openDeletedDataFileScanTask((DeletedDataFileScanTask) task);

    } else {
      throw new IllegalArgumentException(
          "Unsupported changelog scan task type: " + task.getClass().getName());
    }
  }

  CloseableIterable<InternalRow> openAddedRowsScanTask(AddedRowsScanTask task) {
    String filePath = task.file().location();
    SparkDeleteFilter deletes = new SparkDeleteFilter(filePath, task.deletes(), counter(), true);
    return deletes.filter(rows(task, deletes.requiredSchema()));
  }

  private CloseableIterable<InternalRow> openDeletedDataFileScanTask(DeletedDataFileScanTask task) {
    String filePath = task.file().location();

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Avoid changelog mode over tables written with equality/position deletes in the increment range, or compact (rewrite data files) so deletes are applied and removed.
2. Use copy-on-write mode for the writing engine so deletes rewrite files (yielding DeletedDataFileScanTask, which is supported).
3. Run rewrite_delete_files / data compaction actions before reading the increment.
4. Track Iceberg releases — per-row deleted-records changelog support may land in newer versions.

Example fix

// before
spark.readStream.option('stream-from-timestamp', ts).table('tbl') // fails on eq-deletes
// after
spark.sql("CALL system.rewrite_data_files(table => 'db.tbl')")
// or write with copy-on-write so deletes produce whole-file records

// ensure increment range has no unapplied delete files before changelog read
boolean hasDeletes = table.snapshots().stream()
    .filter(s -> s.snapshotId() >= fromSnapshotId)
    .flatMap(s -> s.deleteManifests(table.io()).stream())
    .anyMatch(m -> m.hasDeletedFiles());
if (hasDeletes) runRewriteDeleteFiles();

try { readChangelog(); } catch (UnsupportedOperationException e) {
  if (e.getMessage().contains("Deleted rows scan task")) {
    compactTable(); readChangelog();
  } else throw e;
}

When it happens

Trigger: Reading a changelog/incremental stream (spark.read-stream.format('iceberg').option('stream-scan-interval'...) or batch incremental scan) where a changelog task contains a DeletedRowsScanTask — produced when delete files (position/equality deletes) must be expanded into per-row 'D' records.

Common situations: Streaming changelog reads over tables with v2 equality-delete writes (e.g. Flink CDC writers with upsert mode); merge-on-read tables with delete files in the increment range.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Deleted rows scan task is not supported yet
• Delete files are currently not supported in changelog scans
• Cannot add column %s since setting default values in Spark i
• Transform is not supported: ${transform}
• Cannot convert unknown expression: ${expr}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/277a7d161fed000f. Report an issue: GitHub.