apache/beam · error · RuntimeError

failed to combine histogram.

Error message

failed to combine histogram.

What it means

Histogram.combine only supports merging another Histogram instance using the identical bucket type (linear vs logarithmic). Passing a non-Histogram or a differently-bucketed histogram makes the merge meaningless, so Beam raises RuntimeError. This surfaces in distributed aggregation when partial histograms are combined.

Solutions

1. Ensure all histograms use the same bucket type before combining.
2. Recreate histograms from raw data with a uniform bucket type if mismatched.
3. Convert buckets explicitly (re-bin) before merging if approximation is acceptable.
4. Check isinstance(other, Histogram) and bucket_type equality at call sites.

Example fix

// before
h1 = Histogram(HistogramLinearBuckets(...)); h2 = Histogram(HistogramLogBuckets(...))
h1.combine(h2)
// after
h2 = Histogram(HistogramLinearBuckets(...))  # same bucket type as h1
h1.combine(h2)

from apache_beam.utils.histogram import Histogram
if not isinstance(other, Histogram) or h1._bucket_type != other._bucket_type:
    raise TypeError('histograms must share bucket type')

def can_combine(h1, h2) -> bool:
    return isinstance(h2, Histogram) and h1._bucket_type == h2._bucket_type

try:
    combined = h1.combine(h2)
except RuntimeError:
    combined = None  # rebuild from raw samples with one bucket type

When it happens

Trigger: Calling h1.combine(h2) where h2 is not a Histogram, or h1 uses HistogramLinearBuckets while h2 uses HistogramLogBuckets.

Common situations: Aggregating metrics where different transforms were configured with different bucket strategies; mixing old serialized histograms with new bucket configs after a config change; passing a dict or namedtuple representing histogram data.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• Can not query metrics. Job id is unknown.
• Could not find element
• Could not translate the internal step name %r.
• Could not translate the internal step name %r since job…
• 'Expected GaugeData metric type but received

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f9b991ccbf8fa8e0. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/utils/histogram.py:58

    with self._lock:
      self._buckets = Counter()
      self._num_records = 0
      self._num_top_records = 0
      self._num_bot_records = 0

  def copy(self):
    with self._lock:
      histogram = Histogram(self._bucket_type)
      histogram._num_records = self._num_records
      histogram._num_top_records = self._num_top_records
      histogram._num_bot_records = self._num_bot_records
      histogram._buckets = self._buckets.copy()
      return histogram

  def combine(self, other):
    if not isinstance(other,
                      Histogram) or self._bucket_type != other._bucket_type:
      raise RuntimeError('failed to combine histogram.')
    other_histogram = other.copy()
    with self._lock:
      histogram = Histogram(self._bucket_type)
      histogram._num_records = self._num_records + other_histogram._num_records
      histogram._num_top_records = (
          self._num_top_records + other_histogram._num_top_records)
      histogram._num_bot_records = (
          self._num_bot_records + other_histogram._num_bot_records)
      histogram._buckets = self._buckets + other_histogram._buckets
      return histogram

  def record(self, *args):
    for arg in args:
      self._record(arg)

  def _record(self, value):
    range_from = self._bucket_type.range_from()
    range_to = self._bucket_type.range_to()

View on GitHub (pinned to 12126d8942)