{"record":{"id":"f63e5ce82890e1d6","repo":"apache/beam","slug":"trying-to-return-a-record-starting-at-d-which-is-not","errorCode":null,"errorMessage":"Trying to return a record [starting at %d] which is not greaterthan the last-attempted record [starting at %d]","messagePattern":"Trying to return a record \\[starting at (.+?)\\] which is not greaterthan the last-attempted record \\[starting at (.+?)\\]","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/range_trackers.py","lineNumber":121,"sourceCode":"          'last-returned record [starting at %d]' %\n          (record_start, self._last_record_start))\n\n    if (split_point and self._offset_of_last_split_point != -1 and\n        record_start == self._offset_of_last_split_point):\n      raise ValueError(\n          'Record at a split point has same offset as the previous split '\n          'point: %d' % record_start)\n\n    if not split_point and self._last_record_start == -1:\n      raise ValueError(\n          'The first record [starting at %d] must be at a split point' %\n          record_start)\n\n  def try_claim(self, record_start):\n    with self._lock:\n      # Attempted claim should be monotonous.\n      if record_start <= self._last_attempted_record_start:\n        raise ValueError(\n            'Trying to return a record [starting at %d] which is not greater'\n            'than the last-attempted record [starting at %d]' %\n            (record_start, self._last_attempted_record_start))\n      self._validate_record_start(record_start, True)\n      self._last_attempted_record_start = record_start\n      if record_start >= self.stop_position():\n        return False\n      self._offset_of_last_split_point = record_start\n      self._last_record_start = record_start\n      self._split_points_seen += 1\n      return True\n\n  def set_current_position(self, record_start):\n    with self._lock:\n      self._validate_record_start(record_start, False)\n      self._last_record_start = record_start\n\n  def try_split(self, split_offset):","sourceCodeStart":103,"sourceCodeEnd":139,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/range_trackers.py#L103-L139","documentation":"OffsetRangeTracker.try_claim() enforces that record start positions are strictly monotonic. It raises ValueError when a caller (typically a Beam source's read loop) attempts to claim a record whose start offset is less than or equal to the last attempted record start, which would break progress tracking and resume semantics.","triggerScenarios":"Calling try_claim(record_start) with record_start <= self._last_attempted_record_start, e.g. a custom bounded source re-yielding the same offset, an off-by-one loop that does not advance the iterator, or resuming reads from a stale checkpoint offset.","commonSituations":"Writing a custom iobase.BoundedSource/restriction provider with a buggy loop; re-reading records after a retry without advancing; source splitting code computing overlapping offsets between sub-ranges.","solutions":["Ensure the read loop monotonically advances the offset: only call try_claim with a record_start strictly greater than the previous one.","When resuming, initialize the tracker's start position to the last consumed offset (e.g. from range_tracker.start_position()) so re-reads are not claimed.","Check the offset computation (record size / cursor arithmetic) for an off-by-one that re-emits the same start.","If duplicates are expected from the source, filter/skip records whose start <= last attempted start before claiming."],"exampleFix":"// before\nfor rec in records:\n  if not range_tracker.try_claim(rec.start):\n    break\n  yield rec\n// after\nlast_start = range_tracker.start_position()\nfor rec in records:\n  if rec.start <= last_start:\n    continue  # skip already-attempted records\n  if not range_tracker.try_claim(rec.start):\n    break\n  last_start = rec.start\n  yield rec","handlingStrategy":"validation","validationCode":"if record_start <= range_tracker._last_attempted_record_start:\n  raise ValueError('record_start must be strictly greater than last attempted')","typeGuard":null,"tryCatchPattern":"try:\n  range_tracker.try_claim(record_start)\nexcept ValueError as e:\n  logging.warning('skipping non-monotonic claim: %s', e)","preventionTips":["Always advance the offset cursor before the next try_claim call","Seed iteration from range_tracker.start_position() when resuming","Unit-test custom sources with repeated/retried reads"],"tags":["python","apache-beam","io","range-tracking"],"backgroundTag":"invalid-state-transition","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}