pola-rs/polars · error · ValueError

`offset` input for `with_row_index` cannot be {issue}, got {

Error message

`offset` input for `with_row_index` cannot be {issue}, got {offset}

What it means

LazyFrame.with_row_index(name, offset) propagates an OverflowError from the Rust engine as ValueError when offset is negative or exceeds the maximum index value (unsigned integer bound). The offset seeds the row counter, so it must be a non-negative in-range integer.

Source

Thrown at py-polars/src/polars/lazyframe/frame.py:7510

        ...     pl.all(),
        ... ).collect()
        shape: (3, 3)
        ┌───────┬─────┬─────┐
        │ index ┆ a   ┆ b   │
        │ ---   ┆ --- ┆ --- │
        │ u32   ┆ i64 ┆ i64 │
        ╞═══════╪═════╪═════╡
        │ 0     ┆ 1   ┆ 2   │
        │ 1     ┆ 3   ┆ 4   │
        │ 2     ┆ 5   ┆ 6   │
        └───────┴─────┴─────┘
        """
        try:
            return self._from_pyldf(self._ldf.with_row_index(name, offset))
        except OverflowError:
            issue = "negative" if offset < 0 else "greater than the maximum index value"
            msg = f"`offset` input for `with_row_index` cannot be {issue}, got {offset}"
            raise ValueError(msg) from None

    @deprecated(
        "`LazyFrame.with_row_count` is deprecated; use `LazyFrame.with_row_index` instead."
        " Note that the default column name has changed from 'row_nr' to 'index'."
    )
    def with_row_count(self, name: str = "row_nr", offset: int = 0) -> LazyFrame:
        """
        Add a column at index 0 that counts the rows.

        .. deprecated:: 0.20.4
            Use the :meth:`with_row_index` method instead.
            Note that the default column name has changed from 'row_nr' to 'index'.

        Parameters
        ----------
        name
            Name of the column to add.
        offset

View on GitHub (pinned to df599052da)

Solutions

Clamp: offset = max(0, offset) if a non-negative counter is acceptable
Fix the shard math: use running_total sums of previous shard heights, never differences that can underflow
Validate offset is an int in [0, 2**32) before calling when using default u32 index dtype
Consider chaining an explicit offset column via with_columns if arbitrary integers are needed

Example fix

# before
start = prev_rows - batch  # can be -1 on first batch
lf.with_row_index(offset=start)

# after
start = max(0, prev_rows - batch)
lf.with_row_index(offset=start)

Defensive patterns

Strategy: validation

Validate before calling

offset = int(offset)
if offset < 0:
    raise ValueError(f'row index offset must be >= 0, got {offset}')
lf = lf.with_row_index(name, offset)

Type guard

def is_valid_row_index_offset(offset) -> bool:
    return isinstance(offset, int) and 0 <= offset < 2**32

Try / catch

try:
    lf.with_row_index(offset=offset)
except ValueError:
    lf.with_row_index(offset=max(0, offset))  # or recompute shard offset

Prevention

Compute shard offsets as cumulative sums, never differences
Validate offset range at batch-job start
Unit test batch math with empty first batches

When it happens

Trigger: lf.with_row_index(offset=-1); offsets computed from prior chunk sizes or batch counters that underflow; offsets loaded from config/file parsed as negative; very large offsets beyond u32/u64 bounds after multiprocessing sharding math.

Common situations: Sharded/batched processing where each shard offsets by cumulative row counts (off-by-one or empty-shard math producing -1); user-supplied start indices; migrating from with_row_count which had different validation.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/5308d076d2b9e087. Report an issue: GitHub.