apache/beam · error · ValueError

This function must only be called under the lock self.lock.

Error message

This function must only be called under the lock self.lock.

What it means

_validate_record_start is an internal consistency check of OffsetRangeTracker's ordered-record mode and must be invoked while holding self._lock (as try_claim and set_current_position do). If it is entered without the lock held, the tracker's state could be read/updated concurrently, so it raises ValueError to flag misuse of the internal API.

Solutions

Wrap the call in `with self._lock:` before invoking _validate_record_start.
Reuse try_claim(record_start) / set_current_position(...) instead of calling the private method directly — they acquire the lock correctly.
If subclassing, call super().try_claim(...) rather than reimplementing validation outside the lock.

Example fix

// before
tracker._validate_record_start(5, True)  # no lock held
// after
with tracker._lock:
    tracker._validate_record_start(5, True)

Defensive patterns

Strategy: validation

Validate before calling

assert tracker._lock.locked(), "_validate_record_start requires self._lock held"

Type guard

def lock_held(tracker) -> bool:
    return tracker._lock.locked()

Try / catch

try:
    with tracker._lock:
        tracker._validate_record_start(start, split_point)
except ValueError as e:
    logger.error("Tracker misuse: %s", e)
    raise

Prevention

Never call private tracker methods directly; use try_claim/set_current_position.
Always use `with self._lock:` when subclassing tracker internals.
Add lock-held assertions in custom source tests.

When it happens

Trigger: Calling _validate_record_start (or a custom subclass path) directly from outside a `with self._lock:` block; overriding try_claim/set_current_position in a subclass without acquiring self._lock before validation.

Common situations: Custom Beam IO sources subclassing OffsetRangeTracker and reimplementing claim logic without proper locking; testing internal methods directly without the lock.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c3c37c9c47ed54b4. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/range_trackers.py:97

  @property
  def last_record_start(self):
    return self._last_record_start

  @property
  def last_attempted_record_start(self):
    """Return current value of last_attempted_record_start.

    last_attempted_record_start records a valid position that tried to be
    claimed by calling try_claim(). This value is only updated by `try_claim()`
    no matter `try_claim()` returns `True` or `False`.
    """
    return self._last_attempted_record_start

  def _validate_record_start(self, record_start, split_point):
    # This function must only be called under the lock self.lock.
    if not self._lock.locked():
      raise ValueError(
          'This function must only be called under the lock self.lock.')

    if record_start < self._last_record_start:
      raise ValueError(
          'Trying to return a record [starting at %d] which is before the '
          'last-returned record [starting at %d]' %
          (record_start, self._last_record_start))

    if (split_point and self._offset_of_last_split_point != -1 and
        record_start == self._offset_of_last_split_point):
      raise ValueError(
          'Record at a split point has same offset as the previous split '
          'point: %d' % record_start)

    if not split_point and self._last_record_start == -1:
      raise ValueError(
          'The first record [starting at %d] must be at a split point' %
          record_start)

View on GitHub (pinned to 12126d8942)