apache/beam · error · IllegalStateException

Unknown ChangelogScanTask type: {}

Error message

Unknown ChangelogScanTask type: {}

What it means

ChangelogScanner.analyzeFiles classifies each ChangelogScanTask into insert-tasks or delete-tasks by concrete type (AddedRowsScanTask, DeletedDataFileScanTask, DeletedRowsScanTask). Any other ChangelogScanTask subtype has no analysis strategy, and the scanner throws IllegalStateException — a defensive check against unknown Iceberg task types.

Source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/cdc/ChangelogScanner.java:544

      Schema recIdSchema,
      Comparator<StructLike> idComp) {
    // if table doesn't keep track of metrics, we need to play it safe and consider all tasks may
    // overlap.
    if (!metricsAreAvailable) {
      return AnalysisResult.allBidirectional(tasks);
    }

    List<TaskAndBounds> insertTasks = new ArrayList<>();
    List<TaskAndBounds> deleteTasks = new ArrayList<>();

    try {
      for (ChangelogScanTask task : tasks) {
        if (task instanceof AddedRowsScanTask) {
          insertTasks.add(TaskAndBounds.of(task, recIdSchema, idComp));
        } else if (task instanceof DeletedDataFileScanTask || task instanceof DeletedRowsScanTask) {
          deleteTasks.add(TaskAndBounds.of(task, recIdSchema, idComp));
        } else {
          throw new IllegalStateException("Unknown ChangelogScanTask type: " + task.getClass());
        }
      }
    } catch (TaskAndBounds.NoBoundMetricsException e) {
      // if metrics are not available for some files, we should also play it safe.
      return AnalysisResult.allBidirectional(tasks);
    }

    if (!insertTasks.isEmpty() && !deleteTasks.isEmpty()) {
      Comparator<TaskAndBounds> lowerBoundComp = (t1, t2) -> idComp.compare(t1.lowerId, t2.lowerId);
      Comparator<TaskAndBounds> upperBoundComp = (t1, t2) -> idComp.compare(t1.upperId, t2.upperId);

      insertTasks.sort(lowerBoundComp);
      deleteTasks.sort(lowerBoundComp);

      TaskAndBounds firstInsert = insertTasks.get(0);
      TaskAndBounds firstDelete = deleteTasks.get(0);
      TaskAndBounds lastInsert = insertTasks.stream().max(upperBoundComp).orElseThrow();
      TaskAndBounds lastDelete = deleteTasks.stream().max(upperBoundComp).orElseThrow();

View on GitHub (pinned to 12126d8942)

Solutions

1. Pin/upgrade org.apache.iceberg to the version the Beam CDC connector targets.
2. Disable the new Iceberg table/scan features producing the unknown task type, or use a standard incremental scan instead of the CDC scanner.
3. Patch ChangelogScanner to classify the new task type and rebuild.
4. Check for duplicate/conflicting iceberg-core versions on the classpath (mvn dependency:tree) and de-duplicate.

Example fix

// before
<dependency>org.apache.iceberg:iceberg-core:1.8.0</dependency> // emits unknown task types
// after
<dependency>org.apache.iceberg:iceberg-core:1.5.2</dependency> // matching connector

boolean allKnown = tasks.stream().allMatch(t ->
    t instanceof AddedRowsScanTask || t instanceof DeletedDataFileScanTask || t instanceof DeletedRowsScanTask);
if (!allKnown) throw new IllegalStateException("Incompatible Iceberg task types; check library versions");

boolean isKnownTask(ChangelogScanTask t) {
  return t instanceof AddedRowsScanTask || t instanceof DeletedDataFileScanTask || t instanceof DeletedRowsScanTask;
}

try {
  AnalysisResult r = scanner.result(tasks);
} catch (IllegalStateException e) {
  if (e.getMessage().startsWith("Unknown ChangelogScanTask type")) {
    // fall back to AnalysisResult.allBidirectional(tasks)
  } else { throw e; }
}

When it happens

Trigger: A changelog/incremental scan produces a ChangelogScanTask instance whose type is not among the three handled subtypes (e.g. a new Iceberg task kind introduced in a newer Iceberg version), passed through result() -> analyzeFiles.

Common situations: Iceberg runtime upgraded beyond what the Beam CDC scanner supports; table uses new Iceberg changelog features; mixed library versions on the classpath.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Unknown ChangelogScanTask type: {}
• Expected at least one overlapping task in bidirectional list
• Unsupported task type: {}
• Invalid starting strategy. Valid values are: {values}
• the following options are currently only available when read

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0ef0b792bc5e0d72. Report an issue: GitHub.