pola-rs/polars · error · RuntimeError

cannot create String column without an offsets buffer

Error message

cannot create String column without an offsets buffer

What it means

When ingesting a string column, Polars requires the protocol column's offsets buffer (which delimits variable-length string values). If column.get_buffers()['offsets'] is None, it raises RuntimeError because string boundaries cannot be reconstructed from the data buffer alone. This indicates a broken or incomplete __dataframe__ implementation on the producer side.

Solutions

1. Fix the producer: get_buffers() must return {'data': (buffer, dtype), 'validity': ..., 'offsets': (offsets_buffer, (DtypeKind.INT, 64, 'l', '='))} for string columns.
2. Bypass the producer: convert via a mature path such as pyarrow (obj -> pa.Table -> pl.from_arrow).
3. Report the bug to the producer library if it is third-party.

Example fix

# producer side, before: "offsets": None -> RuntimeError
# after:
def get_buffers(self):
    return {
        "data": (self._chars_buf, self._chars_dtype),
        "validity": self._validity(),
        "offsets": (self._offsets_buf, (DtypeKind.INT, 64, "l", "=")),
    }

def has_offsets(col) -> bool:
    if col.dtype[0] != DtypeKind.STRING:
        return True  # offsets only required for strings
    return col.get_buffers()["offsets"] is not None

bad = [i for i in range(icdf.num_columns()) if not has_offsets(icdf.get_column(i))]
if bad:
    raise ValueError(f"producer bug: string columns {bad} lack an offsets buffer")

try:
    df = pl.from_dataframe(obj)
except RuntimeError as e:
    if "without an offsets buffer" not in str(e):
        raise
    # producer bug: route through pyarrow instead, or fix the producer's get_buffers()

When it happens

Trigger: pl.from_dataframe(obj) where obj is a custom/hand-rolled or buggy third-party interchange object whose string columns return no offsets buffer in get_buffers(); never happens with Polars' own producer, which always supplies offsets.

Common situations: Writing your own interchange adapter for an in-house storage format and forgetting the offsets entry; using an early/unmaintained interop library that returns only data+validity buffers; protocol partial implementations returning None placeholders.

Related errors

• offsets buffer must be cast from
• string buffers must be converted
• bitmask must be constructed
• bitmask must be inverted
• byte-packed boolean buffer must be converted to bit-packed…

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/d8970e0d2d670edf. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/interchange/from_dataframe.py:119

        buffers["validity"], column, dtype, data_buffer, offset, allow_copy=allow_copy
    )
    return pl.Series._from_buffers(dtype, data=data_buffer, validity=validity_buffer)


def _string_column_to_series(column: Column, *, allow_copy: bool) -> Series:
    if column.size() == 0:
        return pl.Series(dtype=String)
    elif not allow_copy:
        msg = "string buffers must be converted"
        raise CopyNotAllowedError(msg)

    buffers = column.get_buffers()
    offset = column.offset

    offsets_buffer_info = buffers["offsets"]
    if offsets_buffer_info is None:
        msg = "cannot create String column without an offsets buffer"
        raise RuntimeError(msg)
    offsets_buffer = _construct_offsets_buffer(
        *offsets_buffer_info, offset, allow_copy=allow_copy
    )

    buffer, dtype = buffers["data"]
    data_buffer = _construct_data_buffer(
        buffer, dtype, buffer.bufsize, offset=0, allow_copy=allow_copy
    )

    # First construct a Series without a validity buffer
    # to allow constructing the validity buffer from a sentinel value
    data_buffers = [data_buffer, offsets_buffer]
    data = pl.Series._from_buffers(String, data=data_buffers, validity=None)

    # Add the validity buffer if present
    validity_buffer = _construct_validity_buffer(
        buffers["validity"], column, String, data, offset, allow_copy=allow_copy
    )

View on GitHub (pinned to 9b5d73fd00)