apache/beam · error · ValueError

target_batch_duration_secs_including_fixed_cost

Error message

target_batch_duration_secs_including_fixed_cost (%s) must be positive

What it means

Raised in _BatchSizeEstimator (BatchElements) __init__ when target_batch_duration_secs_including_fixed_cost is provided but <= 0. Like target_batch_duration_secs, this per-batch wall-clock goal must be strictly positive because it drives the adaptive batch-size search.

Solutions

  1. Pass a strictly positive duration in seconds.
  2. Convert timedeltas via .total_seconds() and verify it is > 0.
  3. Omit the parameter (None) instead of passing 0 when the option should be disabled.

Example fix

// before
beam.BatchElements(target_batch_duration_secs_including_fixed_cost=0)

// after
beam.BatchElements(target_batch_duration_secs_including_fixed_cost=0.5)
Defensive patterns

Strategy: validation

Validate before calling

if target_batch_duration_secs_including_fixed_cost is not None and target_batch_duration_secs_including_fixed_cost <= 0:
    raise ValueError('target_batch_duration_secs_including_fixed_cost must be > 0 seconds')

Type guard

def valid_fixed_cost_duration(x):
    return x is None or (isinstance(x, (int, float)) and x > 0)

Try / catch

try:
    t = beam.BatchElements(target_batch_duration_secs_including_fixed_cost=d)
except ValueError as e:
    logging.error('bad fixed-cost duration %r: %s', d, e)
    raise

Prevention

When it happens

Trigger: BatchElements(target_batch_duration_secs_including_fixed_cost=0) or negative; passing an unconverted timedelta or a computed value that rounded down to 0.

Common situations: Same as the plain duration variant: timedelta passed raw, subtraction yielding 0, config sentinel 0 for 'disabled'.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5eed94ad57bca30e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/util.py:534

      variance=0.25,
      clock=time.time,
      ignore_first_n_seen_per_batch_size=0,
      record_metrics=True):
    if min_batch_size > max_batch_size:
      raise ValueError(
          "Minimum (%s) must not be greater than maximum (%s)" %
          (min_batch_size, max_batch_size))
    if target_batch_overhead and not 0 < target_batch_overhead <= 1:
      raise ValueError(
          "target_batch_overhead (%s) must be between 0 and 1" %
          (target_batch_overhead))
    if target_batch_duration_secs and target_batch_duration_secs <= 0:
      raise ValueError(
          "target_batch_duration_secs (%s) must be positive" %
          (target_batch_duration_secs))
    if (target_batch_duration_secs_including_fixed_cost and
        target_batch_duration_secs_including_fixed_cost <= 0):
      raise ValueError(
          "target_batch_duration_secs_including_fixed_cost "
          "(%s) must be positive" %
          (target_batch_duration_secs_including_fixed_cost))
    if not (target_batch_overhead or target_batch_duration_secs or
            target_batch_duration_secs_including_fixed_cost):
      raise ValueError(
          "At least one of target_batch_overhead or "
          "target_batch_duration_secs or "
          "target_batch_duration_secs_including_fixed_cost must be positive.")
    if ignore_first_n_seen_per_batch_size < 0:
      raise ValueError(
          'ignore_first_n_seen_per_batch_size (%s) must be non '
          'negative' % (ignore_first_n_seen_per_batch_size))
    self._min_batch_size = min_batch_size
    self._max_batch_size = max_batch_size
    self._target_batch_overhead = target_batch_overhead
    self._target_batch_duration_secs = target_batch_duration_secs
    self._target_batch_duration_secs_including_fixed_cost = (

View on GitHub (pinned to 12126d8942)