apache/beam · error · ValueError

At least one of target_batch_overhead or…

Error message

At least one of target_batch_overhead or target_batch_duration_secs or target_batch_duration_secs_including_fixed_cost must be positive.

What it means

Raised in _BatchSizeEstimator (BatchElements) __init__ when none of the three tuning targets (target_batch_overhead, target_batch_duration_secs, target_batch_duration_secs_including_fixed_cost) is set to a positive value. The estimator has no objective to optimize toward, so the library refuses to construct it.

Solutions

1. Supply at least one target, e.g. BatchElements(target_batch_overhead=0.05) or BatchElements(target_batch_duration_secs=1.0).
2. Check keyword spelling of the target_* parameters (a typo yields no error until this check).
3. If targets come from config, validate at load time that at least one is positive.

Example fix

// before
beam.BatchElements(min_batch_size=1, max_batch_size=1000)

// after
beam.BatchElements(min_batch_size=1, max_batch_size=1000,
                   target_batch_overhead=0.05)

if not (target_batch_overhead or target_batch_duration_secs or target_batch_duration_secs_including_fixed_cost):
    raise ValueError('BatchElements requires at least one positive target_batch_* parameter')

def has_batch_target(**kw):
    return any(kw.get(k) for k in ('target_batch_overhead',
                                   'target_batch_duration_secs',
                                   'target_batch_duration_secs_including_fixed_cost'))

try:
    t = beam.BatchElements(min_batch_size=lo, max_batch_size=hi, **targets)
except ValueError as e:
    if 'At least one of' in str(e):
        t = beam.BatchElements(min_batch_size=lo, max_batch_size=hi, target_batch_overhead=0.05)
    else:
        raise

When it happens

Trigger: Calling BatchElements(min_batch_size=..., max_batch_size=...) with all target_* parameters left at defaults (None/0/falsy).

Common situations: Constructing BatchElements with only size bounds, assuming that is enough; passing targets from config where all values were 0/'unset'; typo in a keyword name so the intended target silently stayed None.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• bucket_boundaries must be a non-empty sorted list of…
• bucket_boundaries requires length_fn to be set.
• ignore_first_n_seen_per_batch_size
• Minimum ( ) must not be greater than maximum ( )
• target_batch_duration_secs_including_fixed_cost

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ba71206fa3a3b069. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/util.py:540

          "Minimum (%s) must not be greater than maximum (%s)" %
          (min_batch_size, max_batch_size))
    if target_batch_overhead and not 0 < target_batch_overhead <= 1:
      raise ValueError(
          "target_batch_overhead (%s) must be between 0 and 1" %
          (target_batch_overhead))
    if target_batch_duration_secs and target_batch_duration_secs <= 0:
      raise ValueError(
          "target_batch_duration_secs (%s) must be positive" %
          (target_batch_duration_secs))
    if (target_batch_duration_secs_including_fixed_cost and
        target_batch_duration_secs_including_fixed_cost <= 0):
      raise ValueError(
          "target_batch_duration_secs_including_fixed_cost "
          "(%s) must be positive" %
          (target_batch_duration_secs_including_fixed_cost))
    if not (target_batch_overhead or target_batch_duration_secs or
            target_batch_duration_secs_including_fixed_cost):
      raise ValueError(
          "At least one of target_batch_overhead or "
          "target_batch_duration_secs or "
          "target_batch_duration_secs_including_fixed_cost must be positive.")
    if ignore_first_n_seen_per_batch_size < 0:
      raise ValueError(
          'ignore_first_n_seen_per_batch_size (%s) must be non '
          'negative' % (ignore_first_n_seen_per_batch_size))
    self._min_batch_size = min_batch_size
    self._max_batch_size = max_batch_size
    self._target_batch_overhead = target_batch_overhead
    self._target_batch_duration_secs = target_batch_duration_secs
    self._target_batch_duration_secs_including_fixed_cost = (
        target_batch_duration_secs_including_fixed_cost)
    self._variance = variance
    self._clock = clock
    self._data = []
    self._ignore_next_timing = False
    self._ignore_first_n_seen_per_batch_size = (

View on GitHub (pinned to 12126d8942)