apache/beam · error · ValueError

Current source %r and a source created using the range of…

Error message

Current source %r and a source created using the range of the primary source %r determined by performing dynamic work rebalancing at fraction %r produced different values. Expected these sources to produce the same list of values.

What it means

Thrown by _verify_single_split_fraction_result when the items read from the original source differ from the items read from the primary range produced by split_at_fraction. Dynamic work rebalancing must be value-preserving: the primary range alone must reproduce exactly the items that would have been read up to the split point.

Solutions

Fix the source's RangeTracker so positions map deterministically to record boundaries (position must be the position AFTER returning an element).
Verify read_from_source(source, start, split_stop) yields exactly the first k elements the unsplit read produced.
Make the source deterministic — no reliance on unordered external data during the test.
Use assert_split_at_fraction_binary with a simple deterministic source to isolate whether the RangeTracker or the data is at fault.

Example fix

// before (RangeTracker returns position of record start)
def position_at_fraction(self, fraction):
  return int(fraction * self.stop_position)  # misaligned
// after
def position_at_fraction(self, fraction):
  return self.start_position + int(fraction * (self.stop_position - self.start_position))  # clamp to record boundaries in try_split

Defensive patterns

Strategy: validation

Validate before calling

expected_prefix = items[:k]
primary_only = read_from_source(source, start, split_stop)
assert primary_only == expected_prefix, 'primary range is not value-preserving'

Try / catch

try:
    assert_split_at_fraction_behavior(source, outcome, fraction, k)
except ValueError as e:
    if 'produced different values' in str(e):
        logging.error('Primary range inconsistent with unsplit read: %s', e)

Prevention

Keep RangeTracker positions aligned to record boundaries (position = position after each returned element)
Make test sources fully deterministic
Split-return tests should use the same reader implementation the production pipeline uses

When it happens

Trigger: Calling assert_split_at_fraction_behavior/binary (or the concurrent variant) on a custom source whose RangeTracker produces a primary range whose re-read items don't match the original iteration — typically a broken position<->item mapping in the RangeTracker or a non-deterministic read.

Common situations: Custom BoundedSource whose RangeTracker.start/stop positions don't align with record boundaries; source whose read depends on external mutable state (nondeterministic data); incorrect consumption of split positions after implementing try_split.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/19da477dbc261b9c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/source_test_utils.py:388

    split_successful,
    primary_range,
    residual_range,
    split_fraction):

  assert primary_range
  primary_items = read_from_source(source, *primary_range)

  if not split_successful:
    # For unsuccessful splits, residual_range should be None.
    assert not residual_range

  residual_items = (
      read_from_source(source, *residual_range) if split_successful else [])

  total_items = primary_items + residual_items

  if current_items != primary_items:
    raise ValueError(
        'Current source %r and a source created using the '
        'range of the primary source %r determined '
        'by performing dynamic work rebalancing at fraction '
        '%r produced different values. Expected '
        'these sources to produce the same list of values.' %
        (source, _range_to_str(*primary_range), split_fraction))

  if expected_items != total_items:
    raise ValueError(
        'Items obtained by reading the source %r for primary '
        'and residual ranges %s and %s did not produce the '
        'expected list of values.' %
        (source, _range_to_str(*primary_range), _range_to_str(*residual_range)))

  result = (len(primary_items), len(residual_items) if split_successful else -1)
  return result

View on GitHub (pinned to 12126d8942)