apache/beam · error · ValueError

ignore_first_n_seen_per_batch_size

Error message

ignore_first_n_seen_per_batch_size (%s) must be non negative

What it means

Raised in _BatchSizeEstimator (BatchElements) __init__ when ignore_first_n_seen_per_batch_size is negative. This parameter controls how many initial observations per batch size are ignored for warm-up; a negative count is meaningless and rejected with ValueError.

Solutions

Pass a non-negative integer (0 to disable warm-up skipping).
Clamp the computed value: ignore_n = max(0, computed).
Validate config values before constructing the transform.

Example fix

// before
beam.BatchElements(target_batch_overhead=0.05, ignore_first_n_seen_per_batch_size=-3)

// after
beam.BatchElements(target_batch_overhead=0.05, ignore_first_n_seen_per_batch_size=max(0, ignore_n))

Defensive patterns

Strategy: validation

Validate before calling

if ignore_first_n_seen_per_batch_size < 0:
    raise ValueError('ignore_first_n_seen_per_batch_size must be >= 0')

Type guard

def valid_warmup_skip(n):
    return isinstance(n, int) and n >= 0

Try / catch

try:
    t = beam.BatchElements(target_batch_overhead=0.05, ignore_first_n_seen_per_batch_size=n)
except ValueError:
    t = beam.BatchElements(target_batch_overhead=0.05, ignore_first_n_seen_per_batch_size=max(0, n))

Prevention

Clamp computed warm-up values with max(0, value).
Avoid -1 as a 'disabled' sentinel; use 0 or None semantics explicitly.
Validate the value wherever it is read from config or CLI flags.

When it happens

Trigger: BatchElements(ignore_first_n_seen_per_batch_size=-1) or computing the value from a formula that can go negative.

Common situations: Config arithmetic like desired - observed where observed > desired; a default of -1 used as a sentinel clashing with validation.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/77bc29db3593b402. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/util.py:545

          (target_batch_overhead))
    if target_batch_duration_secs and target_batch_duration_secs <= 0:
      raise ValueError(
          "target_batch_duration_secs (%s) must be positive" %
          (target_batch_duration_secs))
    if (target_batch_duration_secs_including_fixed_cost and
        target_batch_duration_secs_including_fixed_cost <= 0):
      raise ValueError(
          "target_batch_duration_secs_including_fixed_cost "
          "(%s) must be positive" %
          (target_batch_duration_secs_including_fixed_cost))
    if not (target_batch_overhead or target_batch_duration_secs or
            target_batch_duration_secs_including_fixed_cost):
      raise ValueError(
          "At least one of target_batch_overhead or "
          "target_batch_duration_secs or "
          "target_batch_duration_secs_including_fixed_cost must be positive.")
    if ignore_first_n_seen_per_batch_size < 0:
      raise ValueError(
          'ignore_first_n_seen_per_batch_size (%s) must be non '
          'negative' % (ignore_first_n_seen_per_batch_size))
    self._min_batch_size = min_batch_size
    self._max_batch_size = max_batch_size
    self._target_batch_overhead = target_batch_overhead
    self._target_batch_duration_secs = target_batch_duration_secs
    self._target_batch_duration_secs_including_fixed_cost = (
        target_batch_duration_secs_including_fixed_cost)
    self._variance = variance
    self._clock = clock
    self._data = []
    self._ignore_next_timing = False
    self._ignore_first_n_seen_per_batch_size = (
        ignore_first_n_seen_per_batch_size)
    self._batch_size_num_seen = {}
    self._replay_last_batch_size = None
    self._record_metrics = record_metrics
    self._element_count = 0

View on GitHub (pinned to 12126d8942)