apache/beam · error · ValueError

min_batch_size must be >= 1, got {min_batch_size}

Error message

min_batch_size must be >= 1, got {min_batch_size}

What it means

GroupIntoBatches' parameters object validates in __init__ that min_batch_size is at least 1; a batch of size 0 or negative is meaningless so it raises ValueError with the offending value interpolated. This is an eager constructor-time guard, so the failure occurs before any pipeline executes.

Source

Thrown at sdks/python/apache_beam/transforms/util.py:1386

      # Elements are sorted by length and batched optimally

      # Batch with custom size function
      data = [{'text': 'short'}, {'text': 'medium text'},
              {'text': 'long text here'}]
      batched = data | SortAndBatchElements(
          min_batch_size=1,
          max_batch_size=10,
          max_batch_weight=100,
          element_size_fn=lambda x: len(x['text']))
  """
  def __init__(
      self,
      min_batch_size: int,
      max_batch_size: int,
      max_batch_weight: int,
      element_size_fn: Optional[Callable[[Any], int]] = None):
    if min_batch_size < 1:
      raise ValueError(f'min_batch_size must be >= 1, got {min_batch_size}')
    if max_batch_size < min_batch_size:
      raise ValueError(
          f'max_batch_size ({max_batch_size}) must be >= '
          f'min_batch_size ({min_batch_size})')
    if max_batch_weight < 1:
      raise ValueError(f'max_batch_weight must be >= 1, got {max_batch_weight}')
    if element_size_fn is not None and not callable(element_size_fn):
      raise TypeError('element_size_fn must be callable')

    self._min_batch_size = min_batch_size
    self._max_batch_size = max_batch_size
    self._max_batch_weight = max_batch_weight

    # None means the DoFn will use its own _default_element_size method,
    # which tries len() and warns once on TypeError before falling back to 1.
    self._element_size_fn = element_size_fn

  def expand(self, pcoll):

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass an explicit min_batch_size >= 1 to GroupIntoBatches
2. If the value comes from config/CLI, coerce and validate: max(1, int(value))
3. Check upstream defaults so an empty/unset value doesn't fall through as 0

Example fix

// before
util.GroupIntoBatches(min_batch_size=0, max_batch_size=100, max_batch_weight=1000)
// after
util.GroupIntoBatches(min_batch_size=max(1, requested_min), max_batch_size=100, max_batch_weight=1000)

if min_batch_size < 1:
    raise ValueError('min_batch_size must be >= 1')

def valid_min_batch(v): return isinstance(v, int) and v >= 1

When it happens

Trigger: Calling util.GroupIntoBatches(...) with min_batch_size=0 or a negative integer (common when the value comes from a computed/defaulted variable or an unparsed CLI flag).

Common situations: Config values defaulting to 0 via `or` on falsy input; parsing '--min-batch-size' flags without validation; deriving batch size from a division that floors to 0.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• max_batch_size ({max_batch_size}) must be >= min_batch_size
• max_batch_weight must be >= 1, got {max_batch_weight}
• element_size_fn must be callable
• If `num_buckets` is set, it has to be an integer greater tha
• The size parameter must be strictly positive.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/75ca5686bb74fbd8. Report an issue: GitHub.