apache/beam · error · ValueError

target_batch_duration_secs

Error message

target_batch_duration_secs (%s) must be positive

What it means

Raised in _BatchSizeEstimator (BatchElements) __init__ when target_batch_duration_secs is provided but <= 0. The target batch duration is a wall-clock goal per batch and must be a positive number of seconds; zero or negative values are invalid.

Solutions

  1. Pass a positive number of seconds, e.g. target_batch_duration_secs=1.0.
  2. Convert timedeltas explicitly: target_batch_duration_secs=td.total_seconds().
  3. Guard at the call site: if target_duration and target_duration <= 0: raise a clear config error before constructing the transform.
  4. If 0 means 'unset' in your config, omit the argument entirely (None) rather than passing 0.

Example fix

// before
beam.BatchElements(target_batch_duration_secs=timedelta(seconds=2))

// after
beam.BatchElements(target_batch_duration_secs=timedelta(seconds=2).total_seconds())
Defensive patterns

Strategy: validation

Validate before calling

if target_batch_duration_secs is not None and target_batch_duration_secs <= 0:
    raise ValueError('target_batch_duration_secs must be > 0 seconds')

Type guard

def valid_duration_secs(x):
    return x is None or (isinstance(x, (int, float)) and x > 0)

Try / catch

try:
    t = beam.BatchElements(target_batch_duration_secs=d)
except ValueError:
    logging.warning('invalid target_batch_duration_secs=%r; omitting', d)
    t = beam.BatchElements(target_batch_overhead=0.05)

Prevention

When it happens

Trigger: BatchElements(target_batch_duration_secs=0) or a negative value; passing a datetime.timedelta without converting to seconds.

Common situations: Computing the duration from a subtraction that yields 0; passing a timedelta object directly (truthy but invalid) instead of timedelta.total_seconds(); config default of 0 meaning 'unset' colliding with the truthiness check.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4b52dd9d7ea8c98f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/util.py:529

      min_batch_size=1,
      max_batch_size=10000,
      target_batch_overhead=.05,
      target_batch_duration_secs=10,
      target_batch_duration_secs_including_fixed_cost=None,
      variance=0.25,
      clock=time.time,
      ignore_first_n_seen_per_batch_size=0,
      record_metrics=True):
    if min_batch_size > max_batch_size:
      raise ValueError(
          "Minimum (%s) must not be greater than maximum (%s)" %
          (min_batch_size, max_batch_size))
    if target_batch_overhead and not 0 < target_batch_overhead <= 1:
      raise ValueError(
          "target_batch_overhead (%s) must be between 0 and 1" %
          (target_batch_overhead))
    if target_batch_duration_secs and target_batch_duration_secs <= 0:
      raise ValueError(
          "target_batch_duration_secs (%s) must be positive" %
          (target_batch_duration_secs))
    if (target_batch_duration_secs_including_fixed_cost and
        target_batch_duration_secs_including_fixed_cost <= 0):
      raise ValueError(
          "target_batch_duration_secs_including_fixed_cost "
          "(%s) must be positive" %
          (target_batch_duration_secs_including_fixed_cost))
    if not (target_batch_overhead or target_batch_duration_secs or
            target_batch_duration_secs_including_fixed_cost):
      raise ValueError(
          "At least one of target_batch_overhead or "
          "target_batch_duration_secs or "
          "target_batch_duration_secs_including_fixed_cost must be positive.")
    if ignore_first_n_seen_per_batch_size < 0:
      raise ValueError(
          'ignore_first_n_seen_per_batch_size (%s) must be non '
          'negative' % (ignore_first_n_seen_per_batch_size))

View on GitHub (pinned to 12126d8942)