apache/beam · error · IllegalStateException

Unknown ChangelogScanTask type: {}

Error message

Unknown ChangelogScanTask type: {}

What it means

CdcReadUtils.changelogRecordsForTask dispatches each Iceberg ChangelogScanTask to a handler based on task.getType(); the implemented cases do not cover every task type. When a task of an unrecognized type arrives, the default branch throws IllegalStateException.

Source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/cdc/CdcReadUtils.java:235

    Schema outputSchema =
        CdcOutputUtils.readSchemaWithRowMetadata(
            scanConfig.getMetadataColumns(),
            useProjectedSchema ? scanConfig.getRequiredSchema() : table.schema());
    switch (task.getType()) {
      case ADDED_ROWS:
        DeleteFilter<Record> addedDeletesFilter =
            genericDeleteFilter(table, outputSchema, dataFilePath, task.getAddedDeletes());
        return addedDeletesFilter.filter(
            createReader(task, table, scanConfig, addedDeletesFilter.requiredSchema()));
      case DELETED_FILE:
        DeleteFilter<Record> existingDeletesFilter =
            genericDeleteFilter(table, outputSchema, dataFilePath, task.getExistingDeletes());
        return existingDeletesFilter.filter(
            createReader(task, table, scanConfig, existingDeletesFilter.requiredSchema()));
      case DELETED_ROWS:
        return deletedRowsForTask(task, table, scanConfig, outputSchema);
      default:
        throw new IllegalStateException("Unknown ChangelogScanTask type: " + task.getType());
    }
  }

  /**
   * Builds the reader chain for a {@code DELETED_ROWS} task with row-group pushdown when possible.
   * This helps the reader skip entire row groups. For unskipped row groups, the reader should still
   * apply per-record position + equality checks at the row level.
   *
   * <p>We use two pushdown strategies, depending on the type of {@link DeleteFile} in the task
   * (Position Delete vs. Equality Delete). The two strategies can be combined if both {@link
   * DeleteFile} types are present.
   *
   * <ol>
   *   <li><b>Byte-range pushdown for Position Deletes:</b> pre-load the {@link
   *       PositionDeleteIndex}, read the Parquet footer, and compute a single contiguous byte range
   *       covering the row groups that contain at least one deleted position.
   *   <li><b>IN-expression pushdown for Equality Deletes:</b> build an Iceberg {@code IN}
   *       expression and pass it as a Parquet residual so the metrics row-group filter can skip

View on GitHub (pinned to 12126d8942)

Solutions

1. Align the org.apache.iceberg runtime version with the version the Beam connector was built against (upgrade the connector or pin Iceberg).
2. Avoid enabling newer Iceberg table features (unsupported delete formats) on tables read by this pipeline.
3. Patch CdcReadUtils to handle the new task type and rebuild the connector.
4. Inspect task.getType() in logs to identify the exact unhandled type and file an issue upstream.

Example fix

// before
<dependency><groupId>org.apache.iceberg</groupId><artifactId>iceberg-core</artifactId><version>1.7.0</version></dependency> // connector built for 1.5.x
// after
<dependency><groupId>org.apache.iceberg</groupId><artifactId>iceberg-core</artifactId><version>1.5.2</version></dependency>

// pre-flight: verify iceberg runtime matches connector expectation
// e.g. assert CdcReadUtils handles all ChangelogScanTask types present in your scan
Set<String> handled = Set.of("ADDED_ROWS", "DELETED_DATA_FILE", "DELETED_ROWS");
if (!handled.contains(taskType)) throw new IllegalStateException("Unhandled task type: " + taskType);

boolean isHandledTask(ChangelogScanTask t) {
  return t instanceof AddedRowsScanTask || t instanceof DeletedDataFileScanTask || t instanceof DeletedRowsScanTask;
}

try {
  return changelogRecordsForTask(task, table, cfg, schema);
} catch (IllegalStateException e) {
  if (e.getMessage().startsWith("Unknown ChangelogScanTask type")) {
    throw new IllegalStateException("Upgrade Beam Iceberg connector to support task: " + task.getType(), e);
  }
  throw e;
}

When it happens

Trigger: An incremental/changelog scan on an Iceberg table returns a ChangelogScanTask subtype not handled by this reader version (e.g. a newer Iceberg task type like a new delete-kind task), reaching changelogRecordsForTask during CdcRead read expansion.

Common situations: Version mismatch between the Iceberg runtime library and the Beam CDC connector (new Iceberg introduces a task type the connector does not know); reading a table written by newer Iceberg features (e.g. new delete file formats).

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Unknown ChangelogScanTask type: {}
• Unsupported task type: {}
• Invalid starting strategy. Valid values are: {values}
• the following options are currently only available when read
• Unexpected logical type: {}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e7395698cf62314a. Report an issue: GitHub.