apache/beam · error · ValueError

max_batch_weight must be >= 1, got {max_batch_weight}

Error message

max_batch_weight must be >= 1, got {max_batch_weight}

What it means

GroupIntoBatches parameters require max_batch_weight to be a positive integer (>= 1), since a batch carrying zero or negative weight would never close; the constructor raises ValueError with the provided value.

Source

Thrown at sdks/python/apache_beam/transforms/util.py:1392

          min_batch_size=1,
          max_batch_size=10,
          max_batch_weight=100,
          element_size_fn=lambda x: len(x['text']))
  """
  def __init__(
      self,
      min_batch_size: int,
      max_batch_size: int,
      max_batch_weight: int,
      element_size_fn: Optional[Callable[[Any], int]] = None):
    if min_batch_size < 1:
      raise ValueError(f'min_batch_size must be >= 1, got {min_batch_size}')
    if max_batch_size < min_batch_size:
      raise ValueError(
          f'max_batch_size ({max_batch_size}) must be >= '
          f'min_batch_size ({min_batch_size})')
    if max_batch_weight < 1:
      raise ValueError(f'max_batch_weight must be >= 1, got {max_batch_weight}')
    if element_size_fn is not None and not callable(element_size_fn):
      raise TypeError('element_size_fn must be callable')

    self._min_batch_size = min_batch_size
    self._max_batch_size = max_batch_size
    self._max_batch_weight = max_batch_weight

    # None means the DoFn will use its own _default_element_size method,
    # which tries len() and warns once on TypeError before falling back to 1.
    self._element_size_fn = element_size_fn

  def expand(self, pcoll):
    if pcoll.windowing.is_default():
      return pcoll | ParDo(
          _SortAndBatchElementsDoFn(
              self._min_batch_size,
              self._max_batch_size,
              self._max_batch_weight,

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass max_batch_weight >= 1 (estimate from element sizes times target batch size)
2. If you meant unlimited, omit/raise the value rather than passing 0
3. Validate the config value before constructing the transform

Example fix

// before
util.GroupIntoBatches(min_batch_size=1, max_batch_size=100, max_batch_weight=0)
// after
util.GroupIntoBatches(min_batch_size=1, max_batch_size=100, max_batch_weight=1024)

if max_batch_weight < 1:
    raise ValueError('max_batch_weight must be >= 1')

def valid_weight(v): return isinstance(v, int) and v >= 1

When it happens

Trigger: Calling util.GroupIntoBatches with max_batch_weight=0 or negative — often a computed weight limit, a misparsed option, or a literal 0 intended as 'unlimited'.

Common situations: Treating 0 as 'no limit' (the API does not); weight computed from element_size_fn results that underflow; typos like max_batch_weight=-1 in tuning configs.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• min_batch_size must be >= 1, got {min_batch_size}
• max_batch_size ({max_batch_size}) must be >= min_batch_size
• element_size_fn must be callable
• If `num_buckets` is set, it has to be an integer greater tha
• The size parameter must be strictly positive.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0be7627fea580e08. Report an issue: GitHub.