apache/beam · error · ValueError

If `num_buckets` is set, it has to be an integer greater tha

Error message

If `num_buckets` is set, it has to be an integer greater than 0, got %s

What it means

ApproximateUnique (and similar sampling transforms) validate num_buckets: it must be None (use the default) or a positive integer; anything else raises ValueError with the supplied value. Note the constructor first coerces falsy values (0) to the default, but explicit bad values like negatives or non-ints are rejected.

Source

Thrown at sdks/python/apache_beam/transforms/util.py:1624

  transforms.

  Reshuffle adds a temporary random key to each element, performs a
  ReshufflePerKey, and finally removes the temporary key.
  """

  # We use 32-bit integer as the default number of buckets.
  _DEFAULT_NUM_BUCKETS = 1 << 32

  def __init__(self, num_buckets=None):
    """
    :param num_buckets: If set, specifies the maximum random keys that would be
      generated.
    """
    self.num_buckets = num_buckets if num_buckets else self._DEFAULT_NUM_BUCKETS

    valid_buckets = isinstance(num_buckets, int) and num_buckets > 0
    if not (num_buckets is None or valid_buckets):
      raise ValueError(
          'If `num_buckets` is set, it has to be an '
          'integer greater than 0, got %s' % num_buckets)

  def expand(self, pcoll):
    # type: (pvalue.PValue) -> pvalue.PCollection
    if pcoll.pipeline.options.is_compat_version_prior_to(
        RESHUFFLE_TYPEHINT_BREAKING_CHANGE_VERSION):
      reshuffle_step = ReshufflePerKey()
    else:
      reshuffle_step = ReshufflePerKey().with_input_types(
          tuple[int, T]).with_output_types(tuple[int, T])
    return (
        pcoll | 'AddRandomKeys' >>
        Map(lambda t: (random.randrange(0, self.num_buckets), t)
            ).with_input_types(T).with_output_types(tuple[int, T])
        | reshuffle_step
        | 'RemoveRandomKeys' >> Map(lambda t: t[1]).with_input_types(
            tuple[int, T]).with_output_types(T))

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass num_buckets as a positive int (larger for better accuracy) or omit it entirely
2. If reading from config/CLI, convert: num_buckets = int(raw) and check > 0
3. Prefer specifying size (target error) and let num_buckets default

Example fix

// before
ApproximateUnique(num_buckets='1000')
// after
ApproximateUnique(num_buckets=int(config_num_buckets) if config_num_buckets else None)

if num_buckets is not None and not (isinstance(num_buckets, int) and num_buckets > 0):
    raise ValueError('num_buckets must be a positive int or None')

def valid_buckets(v): return v is None or (isinstance(v, int) and not isinstance(v, bool) and v > 0)

When it happens

Trigger: Calling ApproximateUnique(num_buckets=-5), num_buckets=2.5, num_buckets='1000', or num_buckets=0 handled specially but e.g. False; also passing num_buckets alongside size when they conflict in downstream validation.

Common situations: Copy-pasting size values into num_buckets; passing a string from YAML/CLI without int(); using 0 expecting 'auto' when 0 actually becomes the default via falsy coercion but other invalid values raise.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• min_batch_size must be >= 1, got {min_batch_size}
• max_batch_size ({max_batch_size}) must be >= min_batch_size
• max_batch_weight must be >= 1, got {max_batch_weight}
• The size parameter must be strictly positive.
• MatchContinuously interval must be positive.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b5898eeac6a65c42. Report an issue: GitHub.