{"record":{"id":"5c679b8ee0a58aa1","repo":"apache/beam","slug":"start-position-not-0-s","errorCode":null,"errorMessage":"Start position not 0:%s","messagePattern":"Start position not 0:(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/tfrecordio.py","lineNumber":199,"sourceCode":"\nclass _TFRecordSource(FileBasedSource):\n  \"\"\"A File source for reading files of TFRecords.\n\n  For detailed TFRecords format description see:\n    https://www.tensorflow.org/versions/r1.11/api_guides/python/python_io#TFRecords_Format_Details\n  \"\"\"\n  def __init__(self, file_pattern, coder, compression_type, validate):\n    \"\"\"Initialize a TFRecordSource.  See ReadFromTFRecord for details.\"\"\"\n    super().__init__(\n        file_pattern=file_pattern,\n        compression_type=compression_type,\n        splittable=False,\n        validate=validate)\n    self._coder = coder\n\n  def read_records(self, file_name, offset_range_tracker):\n    if offset_range_tracker.start_position():\n      raise ValueError(\n          'Start position not 0:%s' % offset_range_tracker.start_position())\n\n    current_offset = offset_range_tracker.start_position()\n    with self.open_file(file_name) as file_handle:\n      while True:\n        if not offset_range_tracker.try_claim(current_offset):\n          raise RuntimeError('Unable to claim position: %s' % current_offset)\n        record = _TFRecordUtil.read_record(file_handle)\n        if record is None:\n          return  # Reached EOF\n        else:\n          current_offset += _TFRecordUtil.encoded_num_bytes(record)\n          yield self._coder.decode(record)\n\n\ndef _create_tfrecordio_source(\n    file_pattern=None, coder=None, compression_type=None):\n  # We intentionally disable validation for ReadAll pattern so that reading does","sourceCodeStart":181,"sourceCodeEnd":217,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/tfrecordio.py#L181-L217","documentation":"The TFRecord source is not splittable: every file must be read from the very start. If the range tracker's start position is non-zero, read_records raises ValueError to enforce this invariant rather than silently mis-reading records.","triggerScenarios":"Dynamic work rebalancing or a runner attempting to split a TFRecord read at a non-zero offset.","commonSituations":"Custom runners or pipelines calling try_split on unsplittable TFRecord sources; resharding during reads.","solutions":["Read TFRecord sources without splitting (keep splittable=False, the default)","Start the range tracker at position 0 for TFRecord sources","Use a splittable format (e.g. Avro/Parquet) if fine-grained splitting is required"],"exampleFix":"// before\ntracker.try_split(1024)  # mid-file split on TFRecord source\n// after\n# read whole file from offset 0 (default behavior)\nbeam.io.ReadFromTFRecord(pattern)","handlingStrategy":"try-catch","validationCode":"if offset_range_tracker.start_position() not in (0, None):\n    raise ValueError('TFRecord source must start at offset 0')","typeGuard":null,"tryCatchPattern":"try:\n    records = source.read_records(file_name, tracker)\nexcept ValueError as e:\n    if 'Start position not 0' in str(e):\n        tracker = source.get_range_tracker(0, source.DEFAULT_SIZE)\n        records = source.read_records(file_name, tracker)\n    else:\n        raise","preventionTips":["Never call try_split with non-zero positions on unsplittable sources","Keep ReadFromTFRecord default splittable=False","Use splittable formats if dynamic work rebalancing is required"],"tags":["python","beam","tfrecord","splittable"],"backgroundTag":"invalid-state-transition","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}