apache/beam · error · ValueError

Distribution counters support only non-negative value

Error message

Distribution counters support only non-negative value

What it means

The Dataflow Distribution counter only supports non-negative values (it models Google Cloud Dataflow distribution metrics whose buckets assume non-negative inputs). add_input() validates this and raises ValueError for negative elements.

Solutions

Clamp or filter negative values before adding: `counter.add_input(max(0, x))` if semantics allow.
Track negative values separately (e.g. count of negatives in a separate counter) and add absolutes if distribution of magnitude is what matters.
Use a different aggregation (Sum counter, custom accumulator) that supports signed values if negatives are legitimate.

Example fix

# before
for delta in deltas:
  dist.add_input(delta)
# after
for delta in deltas:
  dist.add_input(max(0, delta))

Defensive patterns

Strategy: validation

Validate before calling

if x < 0:
  raise ValueError(f'{x} cannot be added to a distribution counter')

Type guard

def is_non_negative_int(v):
  return isinstance(v, int) and not isinstance(v, bool) and v >= 0

Try / catch

try:
  dist.add_input(value)
except ValueError as e:
  log.warning('Skipping negative distribution input: %s', e)

Prevention

Filter or clamp negatives before metric aggregation.
Track signed quantities with separate sum/min/max counters, not Dataflow distributions.
Add a unit test feeding domain extremes (including negatives) through counter code.

When it happens

Trigger: Calling `DistributionCounter().add_input(x)` with x < 0, e.g. aggregating deltas, signed temperatures, or error codes that can be negative.

Common situations: Metric aggregation code computing diffs (before/after), signed sensor data, or countdown values fed directly into a Dataflow distribution counter.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/621e3fe3c701b3d5. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/py_dataflow_distribution_counter.py:92

  """
  # Assume the max input is sys.maxint, then the possible max bucket size is 59
  MAX_BUCKET_SIZE = 59

  # 3 buckets for every power of ten -> 1, 2, 5
  BUCKET_PER_TEN = 3

  def __init__(self):
    global INT64_MAX  # pylint: disable=global-variable-not-assigned
    self.min = INT64_MAX
    self.max = 0
    self.count = 0
    self.sum = 0
    self.buckets = [0] * self.MAX_BUCKET_SIZE
    self.is_cythonized = False

  def add_input(self, element):
    if element < 0:
      raise ValueError('Distribution counters support only non-negative value')
    self.min = min(self.min, element)
    self.max = max(self.max, element)
    self.count += 1
    self.sum += element
    bucket_index = self.calculate_bucket_index(element)
    self.buckets[bucket_index] += 1

  def add_input_n(self, element, n):
    if element < 0:
      raise ValueError('Distribution counters support only non-negative value')
    self.min = min(self.min, element)
    self.max = max(self.max, element)
    self.count += n
    self.sum += element * n
    bucket_index = self.calculate_bucket_index(element)
    self.buckets[bucket_index] += n

  def calculate_bucket_index(self, element):

View on GitHub (pinned to 12126d8942)