apache/beam · error · IllegalArgumentException

Starting version or timestamp must be specified.

Error message

Starting version or timestamp must be specified.

What it means

CreateCDCReadTasksDoFn resolves the start of a Delta table CDC read. Exactly one of startVersion or startTimestamp must be provided to determine resolvedStartVersion; if neither is set the function cannot compute a starting point and throws. The message reflects the Delta CDC API requirement that a starting version or timestamp be specified.

Source

Thrown at sdks/java/io/delta/src/main/java/org/apache/beam/sdk/io/delta/CreateCDCReadTasksDoFn.java:91

    Configuration conf = new Configuration();
    if (hadoopConfig != null) {
      for (Map.Entry<String, String> entry : hadoopConfig.entrySet()) {
        conf.set(entry.getKey(), entry.getValue());
      }
    }
    Engine engine = DefaultEngine.create(conf);
    Table table = Table.forPath(engine, tablePath);
    TableImpl tableImpl = (TableImpl) table;

    // 1. Resolve starting and ending versions
    long resolvedStartVersion;
    if (startVersion != null) {
      resolvedStartVersion = startVersion;
    } else if (startTimestamp != null) {
      long startMillis = Instant.parse(startTimestamp).toEpochMilli();
      resolvedStartVersion = tableImpl.getVersionAtOrAfterTimestamp(engine, startMillis);
    } else {
      throw new IllegalArgumentException("Starting version or timestamp must be specified.");
    }

    long resolvedEndVersion;
    if (endVersion != null) {
      resolvedEndVersion = endVersion;
    } else if (endTimestamp != null) {
      long endMillis = Instant.parse(endTimestamp).toEpochMilli();
      resolvedEndVersion = tableImpl.getVersionBeforeOrAtTimestamp(engine, endMillis);
    } else {
      resolvedEndVersion = table.getLatestSnapshot(engine).getVersion();
    }

    if (resolvedStartVersion > resolvedEndVersion) {
      throw new IllegalArgumentException(
          String.format(
              "Resolved start version %d is greater than resolved end version %d",
              resolvedStartVersion, resolvedEndVersion));
    }

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass startVersion (long) when you know the version: .startingVersion(5).
2. Otherwise pass a valid ISO-8601 startTimestamp, e.g. .startingTimestamp("2023-01-01T00:00:00Z").
3. Fix option-name typos so the value actually reaches CreateCDCReadTasksDoFn.
4. Validate at pipeline construction time that at least one start bound is present.

Example fix

// before
DeltaCdc.read().tableName("tbl").endingVersion(20) // no start bound
// after
DeltaCdc.read().tableName("tbl").startingVersion(0).endingVersion(20)

if (startVersion == null && startTimestamp == null) {
  throw new IllegalArgumentException(
      "Delta CDC read requires startingVersion or startingTimestamp");
}
if (startTimestamp != null) {
  Instant.parse(startTimestamp); // fail fast on bad format
}

try {
  cdcPipeline.run().waitUntilFinish();
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("Starting version or timestamp must be specified")) {
    // fix the read builder options and relaunch
  }
}

When it happens

Trigger: Building a Delta CDC read (tableChanges/cdc API) with both startVersion and startTimestamp null — e.g. the user set only an ending bound, or the options were misnamed so neither was populated.

Common situations: Option name typos (e.g. 'startingVersion' vs 'startVersion') in the CDC read builder, programmatically built options where the start bound was left unset, migration from a different Delta reader API.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• .exceptionsVia() is required
• Error occurred when consuming changes from Database.
• Resolved start version %d is greater than resolved end versi
• Field must not be null.
• Unsupported change type:

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/71b300da0ca6aed3. Report an issue: GitHub.