apache/beam · error · ValueError

MatchContinuously stop_timestamp %s precedes start_timestamp

Error message

MatchContinuously stop_timestamp %s precedes start_timestamp %s

What it means

When MatchContinuously has a bounded stop_timestamp, _match_deduplicated computes the number of poll windows between start and stop. If stop_timestamp is earlier than start_timestamp the window span is negative and no valid tick plan exists, so Beam raises ValueError during pipeline expansion.

Source

Thrown at sdks/python/apache_beam/io/fileio.py:473

    # Apply windowing last because dedup relies on the global window.
    if self.apply_windowing:
      match_files = match_files | beam.WindowInto(FixedWindows(self.interval))

    return match_files

  def _match_deduplicated(self,
                          pbegin) -> beam.PCollection[filesystem.FileMetadata]:
    # Watch emits each file once per dedup key: the path, joined by the mtime
    # when matching updated files. stop_timestamp bounds the polls to
    # [start, stop).
    clock = _PollClock()
    if self.stop_ts == MAX_TIMESTAMP:
      termination = never()
    else:
      start_ts = Timestamp.of(self.start_ts)
      stop_ts = Timestamp.of(self.stop_ts)
      if stop_ts < start_ts:
        raise ValueError(
            'MatchContinuously stop_timestamp %s precedes start_timestamp %s' %
            (stop_ts, start_ts))
      interval_micros = Duration.of(self.interval).micros
      span_micros = (stop_ts - start_ts).micros
      # Ceiling division reproduces PeriodicImpulse's tick count; the window
      # upper bound is exclusive.
      max_polls = -(-span_micros // interval_micros)
      if max_polls == 0:
        # An empty [start, stop) window never ticks; the impulse path keeps
        # the output empty without Watch's unconditional first poll.
        return self._match_all_each_poll(pbegin)
      termination = _WatchWindowTermination(clock, start_ts.micros, max_polls)
    poll_fn = _MatchContinuouslyPollFn(
        self.empty_match_treatment,
        self.start_ts,
        clock,
        mtime_timestamps=self.timestamp_cursor)
    # The key coder is inferred from the key function's return annotation.

View on GitHub (pinned to 12126d8942)

Solutions

Ensure stop_timestamp is later than start_timestamp; swap the values if they were passed in the wrong order.
If you want an unbounded match, omit stop_timestamp (defaults to MAX_TIMESTAMP / never terminates).
Add an assertion or min/max normalization on the two timestamps before constructing the transform.

Example fix

# before
MatchContinuously('/data/*', start_timestamp=now, stop_timestamp=now - timedelta(hours=1))
# after
start, stop = now - timedelta(hours=1), now
MatchContinuously('/data/*', start_timestamp=start, stop_timestamp=stop)

Defensive patterns

Strategy: validation

Validate before calling

if stop_ts is not None and stop_ts < start_ts:
    raise ValueError('stop_timestamp must be >= start_timestamp')

Prevention

Compute start/stop from a single 'now' reference to avoid ordering bugs.
Normalize with start, stop = min(a,b), max(a,b) when inputs may be unordered.

When it happens

Trigger: Constructing MatchContinuously(..., start_timestamp=ts1, stop_timestamp=ts2) where ts2 < ts1, then running expand() (has_deduplication path).

Common situations: Computing start/stop times from relative offsets ('now - 1h' as stop vs 'now' as start), swapping the arguments, or timezone confusion causing the stop time to precede the start.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8008a4355750d028. Report an issue: GitHub.