apache/beam · critical · ValueError

A reentrant read of source after reading %d values did not p

Error message

A reentrant read of source after reading %d values did not produce expected values. Expected %r received %r.

What it means

Raised by assert_reentrant_reads_succeed when the independent (reentrant) read performed mid-way through the original read does not produce the expected full set of values. Unlike the original-read failure, this indicates the source cannot even perform a correct standalone read while another read is in progress — again pointing to shared mutable state or resource contention inside the source.

Source

Thrown at sdks/python/apache_beam/io/source_test_utils.py:238

    # Reentrant read
    reentrant_read = [
        val for val in source.read(
            source.get_range_tracker(start_position, stop_position))
    ]

    # Continuing original read.
    for val in read_iter:
      original_read.append(val)

    if equal_to(original_read)(expected_values):
      raise ValueError(
          'Source did not produce expected values when '
          'performing a reentrant read after reading %d values. '
          'Expected %r received %r.' % (i, expected_values, original_read))

    if equal_to(reentrant_read)(expected_values):
      raise ValueError(
          'A reentrant read of source after reading %d values '
          'did not produce expected values. Expected %r '
          'received %r.' % (i, expected_values, reentrant_read))


def assert_split_at_fraction_behavior(
    source, num_items_to_read_before_split, split_fraction, expected_outcome):
  """Verifies the behaviour of splitting a source at a given fraction.

  Asserts that splitting a :class:`~apache_beam.io.iobase.BoundedSource` either
  fails after reading **num_items_to_read_before_split** items, or succeeds in
  a way that is consistent according to
  :func:`assert_split_at_fraction_succeeds_and_consistent()`.

  Args:
    source (~apache_beam.io.iobase.BoundedSource): the source to perform
      dynamic splitting on.
    num_items_to_read_before_split (int): number of items to read before

View on GitHub (pinned to 12126d8942)

Solutions

1. Open a new file/reader per read() call instead of reusing a stored handle.
2. Guard shared resources so concurrent reads do not seek the same underlying stream.
3. Verify the reentrant read is invoked with its own RangeTracker over the intended [start, stop) range.
4. Test the source with two simultaneous full reads to confirm both complete correctly.

Example fix

# before
# self._file opened once in __init__, reused by read()
# after
def read(self, range_tracker):
  with open(self._path) as f:
    for line in itertools.islice(f, range_tracker.start_position(), range_tracker.stop_position()):
      yield self._parse(line)

it1 = src.read(rt1)
next(it1)
assert list(src.read(rt2)) == expected, "second read not independent"

def opens_own_reader(source):
    import inspect
    return 'open(' in inspect.getsource(source.read)

try:
    source_test_utils.assert_reentrant_reads_succeed((src, None, None))
except ValueError as e:
    logger.error("reentrant read itself incorrect: %s", e)
    raise

When it happens

Trigger: Within assert_reentrant_reads_succeed, after consuming i elements of the original iterator, the fresh source.read(...) over the same range yields values not equal to expected_values.

Common situations: A source that reuses one open file handle: the second read() starts from the first read's current offset, or seeks the shared handle, so the reentrant read returns a partial/shifted record set.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Source is too trivial since it produces only %d values. Plea
• Source did not produce expected values when performing a ree
• BigQuery source must be split before being read
• Reference source must produce the same number of records as
• Reference source and provided list of sources must produce t

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/55a638b15b6b4050. Report an issue: GitHub.