apache/beam · error · ValueError

Watch allowed_lateness must not be negative, got %s

Error message

Watch allowed_lateness must not be negative, got %s

What it means

Watch's allowed_lateness controls how long output produced after the watermark is still accepted. It is converted with _as_duration and validated to be non-negative; a negative Duration (or a negative convertible value) is rejected at construction time.

Source

Thrown at sdks/python/apache_beam/io/watch.py:861

    now_fn: clock used for termination decisions; tests can inject one.
  """
  def __init__(
      self,
      poll_fn: Callable[[Any], PollResult],
      poll_interval,
      termination: Optional[TerminationCondition] = None,
      output_coder: Optional[Coder] = None,
      output_key_fn: Optional[Callable[[Any], Any]] = None,
      output_key_coder: Optional[Coder] = None,
      timestamp_cursor: bool = False,
      allowed_lateness=0,
      now_fn: Optional[Callable[[], float]] = None):
    super().__init__()
    if poll_interval is None:
      raise ValueError('Watch requires a poll_interval')
    allowed_lateness = _as_duration(allowed_lateness)
    if allowed_lateness < Duration(0):
      raise ValueError(
          'Watch allowed_lateness must not be negative, got %s' %
          allowed_lateness)
    self._poll_fn = poll_fn
    self._poll_interval = _as_duration(poll_interval)
    self._termination = termination or never()
    self._output_coder = output_coder
    self._output_key_fn = output_key_fn
    self._output_key_coder = output_key_coder
    self._timestamp_cursor = timestamp_cursor
    self._allowed_lateness = allowed_lateness
    self._now = now_fn

  def expand(self, pcoll):
    output_coder = self._output_coder
    if output_coder is None and isinstance(self._poll_fn, PollFn):
      output_coder = self._poll_fn.default_output_coder()
    if output_coder is None:
      output_coder = _coder_for_hint(_poll_output_type(self._poll_fn))

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass a non-negative duration, e.g. allowed_lateness=0 (default) or a positive value like allowed_lateness='1h'
2. Clamp or validate the config value before passing: max(0, configured_lateness)
3. If negative lateness was intended to disable lateness handling, use 0 instead

Example fix

// before
beam.Watch(poll_fn, poll_interval=10, allowed_lateness=-60)
// after
beam.Watch(poll_fn, poll_interval=10, allowed_lateness=60)

lateness = _as_duration(allowed_lateness)
if lateness < Duration(0):
    raise ValueError('allowed_lateness must be >= 0')

def is_valid_lateness(v) -> bool:
    from apache_beam.utils.timestamp import Duration
    try:
        return _as_duration(v) >= Duration(0)
    except Exception:
        return False

When it happens

Trigger: Constructing beam.Watch or _Pollers with allowed_lateness=-5, a negative timedelta, or a negative string like '-10s' that _as_duration converts to a negative Duration.

Common situations: Configuration sign errors (subtracting lateness instead of adding); confusion with APIs where negative offsets mean something; loading negative values from pipeline options or YAML configs.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• op must be one of ('idxmax', 'idxmin'). got {op!r}.
• align_axis must be one of ('index', 0, 'columns', 1). got {a
• unknown Watch growth state tag: %r
• Watch restriction was neither claimed nor checkpointed: %r
• Watch requires a poll_interval

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2e7d78fb462ee832. Report an issue: GitHub.