apache/beam · error · ValueError

ApproximateUnique needs a size >= 16 for an error <= 0.50…

Error message

ApproximateUnique needs a size >= 16 for an error <= 0.50. In general, the estimation error is about 2 / sqrt(sample_size). Received {size = %s}.

What it means

When size is given, parse_input_params requires an int >= 16 because the estimation error is roughly 2/sqrt(sample_size); smaller samples exceed 0.50 error. A non-int or size < 16 triggers _INPUT_SIZE_ERR_MSG. (Note: strings are also rejected since isinstance(size, int) is checked.)

Solutions

1. Use an integer sample size >= 16, sized so 2/sqrt(size) meets your error tolerance (e.g. size=10000 gives ~2% error).
2. Convert config strings to int before passing (int(value)).
3. Prefer specifying error (0.01–0.50) and let _get_sample_size_from_est_error compute the sample size.

Example fix

// before
beam.ApproximateUnique(size='100')
// after
beam.ApproximateUnique(size=10000)  # int >= 16

if size is not None:
    assert isinstance(size, int) and not isinstance(size, bool) and size >= 16, \
        f'size must be int >= 16, got {size!r}'

def is_valid_sample_size(size) -> bool:
    return isinstance(size, int) and not isinstance(size, bool) and size >= 16

try:
    t = beam.ApproximateUnique(size=size)
except ValueError as e:
    if 'size >= 16' in str(e):
        t = beam.ApproximateUnique(size=max(16, int(size) if size else 16))
    else:
        raise

When it happens

Trigger: beam.ApproximateUnique(size=10); beam.ApproximateUnique(size='1000') (string from config); size as float like 100.0 in strict type paths.

Common situations: Loading numeric options from CLI/JSON where they arrive as strings; users underestimating the minimum sample; copying examples with tiny sample sizes.

Related errors

• ApproximateUnique needs an estimation error between 0.01…
• Cannot set position to
• Either size or error should be set. Received
• Either size or error should be set. Received
• PartitionFn specified out-of-bounds partition index

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ed2d6dd64822211c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/stats.py:127

    :param size: an int not smaller than 16, which we would use to estimate
      number of unique values.
    :param error: max estimation error, which is a float between 0.01 and 0.50.
      If error is given, sample size will be calculated from error with
      _get_sample_size_from_est_error function.
    :return: sample size
    :raises:
      ValueError: If both size and error are given, or neither is given, or
      values are out of range.
    """

    if None not in (size, error):
      raise ValueError(ApproximateUnique._MULTI_VALUE_ERR_MSG % (size, error))
    elif size is None and error is None:
      raise ValueError(ApproximateUnique._NO_VALUE_ERR_MSG)
    elif size is not None:
      if not isinstance(size, int) or size < 16:
        raise ValueError(ApproximateUnique._INPUT_SIZE_ERR_MSG % (size))
      else:
        return size
    else:
      if error < 0.01 or error > 0.5:
        raise ValueError(ApproximateUnique._INPUT_ERROR_ERR_MSG % (error))
      else:
        return ApproximateUnique._get_sample_size_from_est_error(error)

  @staticmethod
  def _get_sample_size_from_est_error(est_err):
    """
    :return: sample size

    Calculate sample size from estimation error
    """
    return math.ceil(4.0 / math.pow(est_err, 2.0))

  @typehints.with_input_types(T)

View on GitHub (pinned to 12126d8942)