pola-rs/polars · error · RuntimeError

cannot create String column without an offsets buffer

Error message

cannot create String column without an offsets buffer

What it means

Raised when a String column coming from an interchange producer reports no offsets buffer (buffers['offsets'] is None). The protocol requires string columns to provide both a data buffer and an offsets buffer; without offsets, element boundaries are unknown and a String Series cannot be constructed. This always indicates a malformed or non-conforming producer implementation.

Source

Thrown at py-polars/src/polars/interchange/from_dataframe.py:119

        buffers["validity"], column, dtype, data_buffer, offset, allow_copy=allow_copy
    )
    return pl.Series._from_buffers(dtype, data=data_buffer, validity=validity_buffer)


def _string_column_to_series(column: Column, *, allow_copy: bool) -> Series:
    if column.size() == 0:
        return pl.Series(dtype=String)
    elif not allow_copy:
        msg = "string buffers must be converted"
        raise CopyNotAllowedError(msg)

    buffers = column.get_buffers()
    offset = column.offset

    offsets_buffer_info = buffers["offsets"]
    if offsets_buffer_info is None:
        msg = "cannot create String column without an offsets buffer"
        raise RuntimeError(msg)
    offsets_buffer = _construct_offsets_buffer(
        *offsets_buffer_info, offset, allow_copy=allow_copy
    )

    buffer, dtype = buffers["data"]
    data_buffer = _construct_data_buffer(
        buffer, dtype, buffer.bufsize, offset=0, allow_copy=allow_copy
    )

    # First construct a Series without a validity buffer
    # to allow constructing the validity buffer from a sentinel value
    data_buffers = [data_buffer, offsets_buffer]
    data = pl.Series._from_buffers(String, data=data_buffers, validity=None)

    # Add the validity buffer if present
    validity_buffer = _construct_validity_buffer(
        buffers["validity"], column, String, data, offset, allow_copy=allow_copy
    )

View on GitHub (pinned to df599052da)

Solutions

1. Fix the producer: return a (buffer, dtype) tuple for buffers['offsets'] for every String column
2. Validate the producer against the reference interchange consumer tests before integration
3. If the producer cannot be fixed, bypass the protocol and move the data through Arrow (pl.from_arrow)

def producer_string_columns_have_offsets(df) -> bool:
    proto = df.__dataframe__(allow_copy=False)
    for col in proto.get_columns():
        if col.dtype[0] == 21 and col.size() > 0:  # 21 == DtypeKind.STRING
            if col.get_buffers()['offsets'] is None:
                return False
    return True

try:
    out = pl.from_dataframe(df)
except RuntimeError as e:
    if 'without an offsets buffer' in str(e):
        # producer is non-conforming; bypass interchange
        out = pl.from_arrow(df.to_arrow())
    else:
        raise

When it happens

Trigger: A custom or third-party __dataframe__ implementation whose Column.get_buffers() returns None for the 'offsets' key of a non-empty String column.

Common situations: Writing your own interchange producer and forgetting the offsets buffer; using an early or minimal producer backend; protocol spec drift between producer and polars versions.

Related errors

• string buffers must be converted
• string buffers must be converted
• non-dictionary categoricals are not yet supported
• offsets buffer must be cast from {polars_dtype} to Int64
• invalid sentinel value for column of type {column_dtype}: {n

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/d8970e0d2d670edf. Report an issue: GitHub.