apache/beam · error · ValueError

Cannot set position to

Error message

Cannot set position to %d since it's larger than size of data %d.

What it means

The in-memory _Position class used by textio sources asserts the position set is within the length of the in-memory data buffer, raising this ValueError otherwise. It protects against readers advancing past the end of the backing byte data.

Solutions

Clamp the new position to len(self._data) before assigning: value = min(value, len(data)).
Use position == len(data) (not larger) to represent end-of-buffer.
Refresh self._data before setting position if the buffer was reset or replaced.
Set position via the source's own advance logic (read/skip) instead of writing .position directly.

Example fix

// before
pos.position = len(new_data) + 1
// after
pos.position = min(len(new_data), requested_position)

Defensive patterns

Strategy: validation

Validate before calling

def safe_set_position(pos, value, data):
    assert isinstance(value, int)
    pos.position = min(value, len(data))

Type guard

def is_valid_position(value, data):
    return isinstance(value, int) and 0 <= value <= len(data)

Try / catch

try:
    pos.position = value
except (ValueError, AssertionError) as e:
    logging.error('position out of range: %s', e)
    pos.position = len(data)

Prevention

Clamp positions to len(data) before assignment
Represent EOF with position == len(data), never len(data)+1
Re-sync position objects after resetting/replacing the data buffer

When it happens

Trigger: Setting .position on the position object to a value greater than len(self._data); typically inside custom ReadProgress/skip logic or tests that manipulate the position directly, e.g. after the buffer was replaced with shorter data.

Common situations: Custom text source subclass computing offsets from the original (uncompressed/larger) data against a truncated buffer; tests reusing a position object after resetting data to b''; off-by-one when setting position to len(data)+1 to signal EOF.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a9d75cf1ab620b8f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/textio.py:98

    @property
    def data(self):
      return self._data

    @data.setter
    def data(self, value):
      assert isinstance(value, bytes)
      self._data = value

    @property
    def position(self):
      return self._position

    @position.setter
    def position(self, value):
      assert isinstance(value, int)
      if value > len(self._data):
        raise ValueError(
            'Cannot set position to %d since it\'s larger than '
            'size of data %d.' % (value, len(self._data)))
      self._position = value

    def reset(self):
      self.data = b''
      self.position = 0

  def __init__(
      self,
      file_pattern,
      min_bundle_size,
      compression_type,
      strip_trailing_newlines,
      coder: coders.Coder,
      buffer_size=DEFAULT_READ_BUFFER_SIZE,
      validate=True,
      skip_header_lines=0,

View on GitHub (pinned to 12126d8942)