apache/beam · error · IllegalArgumentException

Resolved start version %d is greater than resolved end versi

Error message

Resolved start version %d is greater than resolved end version %d

What it means

Thrown by CreateCDCReadTasksDoFn.processElement after resolving the table start and end versions (from timestamps or latest snapshot). Delta change-data-feed reads require a monotonically increasing version interval; if the resolved start version exceeds the resolved end version the requested time range is invalid and the read cannot proceed.

Source

Thrown at sdks/java/io/delta/src/main/java/org/apache/beam/sdk/io/delta/CreateCDCReadTasksDoFn.java:105

    } else if (startTimestamp != null) {
      long startMillis = Instant.parse(startTimestamp).toEpochMilli();
      resolvedStartVersion = tableImpl.getVersionAtOrAfterTimestamp(engine, startMillis);
    } else {
      throw new IllegalArgumentException("Starting version or timestamp must be specified.");
    }

    long resolvedEndVersion;
    if (endVersion != null) {
      resolvedEndVersion = endVersion;
    } else if (endTimestamp != null) {
      long endMillis = Instant.parse(endTimestamp).toEpochMilli();
      resolvedEndVersion = tableImpl.getVersionBeforeOrAtTimestamp(engine, endMillis);
    } else {
      resolvedEndVersion = table.getLatestSnapshot(engine).getVersion();
    }

    if (resolvedStartVersion > resolvedEndVersion) {
      throw new IllegalArgumentException(
          String.format(
              "Resolved start version %d is greater than resolved end version %d",
              resolvedStartVersion, resolvedEndVersion));
    }

    // 2. Load snapshot at resolvedEndVersion to get the scanStateRow
    // We use endVersion's schema because it represents the latest schema in the
    // read range
    // which handles schema evolution (older files will just lack new columns).
    Snapshot endSnapshot = table.getSnapshotAsOfVersion(engine, resolvedEndVersion);
    Scan scan = endSnapshot.getScanBuilder().build();
    Row scanState = scan.getScanState(engine);
    SerializableRow serializableScanState = new SerializableRow(scanState);

    // 3. Load snapshot at resolvedStartVersion to initialize the CommitRange
    Snapshot startSnapshot = table.getSnapshotAsOfVersion(engine, resolvedStartVersion);

    CommitRangeBuilder rangeBuilder =

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify startTimestamp <= endTimestamp (and both are epoch millis in UTC) at pipeline-construction time before submitting the job.
2. Clamp the start timestamp to the table's earliest version, or use version-based reads instead of timestamps for deterministic bounds.
3. Check for timezone/DST conversion bugs in the code that produces the millis values (e.g. date parsed in local time vs UTC).
4. If the end timestamp is in the future, note Delta resolves it to the latest snapshot; ensure the start resolves to an earlier version or pick an earlier start.

Example fix

// before
Long start = parseLocal("2026-09-01"); // timezone bug
Long end = parseUtc("2026-08-01");
DeltaIO.read().readChanges().withStartTimestamp(start).withEndTimestamp(end);
// after
long start = parseUtcInstant("2026-08-01").toEpochMilli();
long end = parseUtcInstant("2026-09-01").toEpochMilli();
if (start > end) throw new IllegalArgumentException("start must be <= end");
DeltaIO.read().readChanges().withStartTimestamp(start).withEndTimestamp(end);

if (startMillis > endMillis) throw new IllegalArgumentException("start timestamp must be <= end timestamp (got " + startMillis + " > " + endMillis + ")");

When it happens

Trigger: Calling DeltaIO.readChanges with a start timestamp later than the end timestamp, or a start timestamp/version that resolves to a version after the table's latest snapshot (e.g. a future timestamp, or an end 'timestamp' of 0/epoch).

Common situations: Swapped start/end parameters; using local-clock timestamps against a table written with a different clock skew; asking for changes since a timestamp after the last commit; DST/timezone mistakes converting wall-clock time to epoch millis.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Unsupported metadata column %s. Supported columns are: %s, %
• Starting version or timestamp must be specified.
• Field must not be null.
• Unsupported change type:
• Iterable at position ${i} was null.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9cc44fb65ea43240. Report an issue: GitHub.