apache/beam · error · ValueError

MatchContinuously interval must be positive.

Error message

MatchContinuously interval must be positive.

What it means

MatchContinuously polls a filesystem at a fixed interval; expand() validates this interval at pipeline construction time. A zero or negative interval would make the periodic match impulse invalid or produce no ticks, so Beam raises ValueError before running the pipeline.

Source

Thrown at sdks/python/apache_beam/io/fileio.py:449

      if not has_deduplication:
        raise ValueError(
            'MatchContinuously(timestamp_cursor=True) deduplicates, so it '
            'requires has_deduplication=True.')
      if not match_updated_files:
        _LOGGER.warning(
            'MatchContinuously(timestamp_cursor=True) implies '
            'match_updated_files=True.')
        self.match_upd = True
    else:
      _LOGGER.warning(
          'Matching Continuously is stateful, and can scale poorly. '
          'Consider using Pub/Sub Notifications '
          '(https://cloud.google.com/storage/docs/pubsub-notifications) '
          'if possible')

  def expand(self, pbegin) -> beam.PCollection[filesystem.FileMetadata]:
    if Duration.of(self.interval).micros <= 0:
      raise ValueError('MatchContinuously interval must be positive.')
    if self.has_deduplication:
      match_files = self._match_deduplicated(pbegin)
    else:
      match_files = self._match_all_each_poll(pbegin)

    # Apply windowing last because dedup relies on the global window.
    if self.apply_windowing:
      match_files = match_files | beam.WindowInto(FixedWindows(self.interval))

    return match_files

  def _match_deduplicated(self,
                          pbegin) -> beam.PCollection[filesystem.FileMetadata]:
    # Watch emits each file once per dedup key: the path, joined by the mtime
    # when matching updated files. stop_timestamp bounds the polls to
    # [start, stop).
    clock = _PollClock()
    if self.stop_ts == MAX_TIMESTAMP:

View on GitHub (pinned to 12126d8942)

Solutions

Pass a positive interval, e.g. interval=30 (seconds) or a positive datetime.timedelta.
Validate/normalize the interval value before constructing the transform, clamping to a positive minimum.
Check units: Beam Durations are in seconds/Duration objects; ensure the computation isn't truncating to 0.

Example fix

# before
MatchContinuously('/data/*', interval=interval_from_config)  # 0
# after
interval = max(interval_from_config, 1)
MatchContinuously('/data/*', interval=interval)

Defensive patterns

Strategy: validation

Validate before calling

from apache_beam.utils.windowed_value import Duration
assert Duration.of(interval).micros > 0, 'interval must be positive'

Prevention

Clamp dynamic interval values to a positive minimum.
Prefer explicit datetime.timedelta(seconds=...) literals over computed integers.

When it happens

Trigger: Calling MatchContinuously(..., interval=0), a negative timedelta/Duration, or expand() on a transform built with such an interval.

Common situations: Computing an interval dynamically (e.g. from config or a division that yields 0) and passing it unvalidated; mistaking interval units (seconds vs microseconds) resulting in 0.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8d08a940f2776b8c. Report an issue: GitHub.