pola-rs/polars · error · CopyNotAllowedError

string buffers must be converted

Error message

string buffers must be converted

What it means

Raised by _string_column_to_series when converting a non-empty String column with allow_copy=False. The interchange protocol represents strings as separate offsets and data buffers; Polars must repack them into its own binary string layout, and that always allocates. Only an empty string column (size 0) is accepted without copying.

Source

Thrown at py-polars/src/polars/interchange/from_dataframe.py:111

) -> Series:
    buffers = column.get_buffers()
    offset = column.offset

    data_buffer = _construct_data_buffer(
        *buffers["data"], column.size(), offset, allow_copy=allow_copy
    )
    validity_buffer = _construct_validity_buffer(
        buffers["validity"], column, dtype, data_buffer, offset, allow_copy=allow_copy
    )
    return pl.Series._from_buffers(dtype, data=data_buffer, validity=validity_buffer)


def _string_column_to_series(column: Column, *, allow_copy: bool) -> Series:
    if column.size() == 0:
        return pl.Series(dtype=String)
    elif not allow_copy:
        msg = "string buffers must be converted"
        raise CopyNotAllowedError(msg)

    buffers = column.get_buffers()
    offset = column.offset

    offsets_buffer_info = buffers["offsets"]
    if offsets_buffer_info is None:
        msg = "cannot create String column without an offsets buffer"
        raise RuntimeError(msg)
    offsets_buffer = _construct_offsets_buffer(
        *offsets_buffer_info, offset, allow_copy=allow_copy
    )

    buffer, dtype = buffers["data"]
    data_buffer = _construct_data_buffer(
        buffer, dtype, buffer.bufsize, offset=0, allow_copy=allow_copy
    )

    # First construct a Series without a validity buffer

View on GitHub (pinned to df599052da)

Solutions

1. Call pl.from_dataframe(df, allow_copy=True) for data that contains strings
2. If zero-copy is mandatory, drop or transform string columns upstream before conversion
3. Wrap the call in try/except CopyNotAllowedError and retry with allow_copy=True so only converting inputs pay the copy

Example fix

// before
df = pl.from_dataframe(producer_df, allow_copy=False)  # CopyNotAllowedError on strings

// after
try:
    df = pl.from_dataframe(producer_df, allow_copy=False)
except pl.exceptions.CopyNotAllowedError:
    df = pl.from_dataframe(producer_df, allow_copy=True)

def has_nonempty_string_column(df) -> bool:
    proto = df.__dataframe__(allow_copy=False)
    from polars.interchange.protocol import DtypeKind
    for col in proto.get_columns():
        if col.dtype[0] == DtypeKind.STRING and col.size() > 0:
            return True
    return False

# if True, expect a copy; pass allow_copy=True directly

try:
    out = pl.from_dataframe(df, allow_copy=False)
except pl.exceptions.CopyNotAllowedError:
    # string layout must be repacked; retry with copies allowed
    out = pl.from_dataframe(df, allow_copy=True)

When it happens

Trigger: pl.from_dataframe(df, allow_copy=False) where df contains at least one non-empty string column (offsets buffer + UInt8 data buffer).

Common situations: Zero-copy pipelines (feature serving, shared-memory handoff, memory-budgeted ingest) that set allow_copy=False globally; consuming from pandas/ibis/cudf/custom producers that expose string data through the interchange protocol.

Related errors

• offsets buffer must be cast from {polars_dtype} to Int64
• non-contiguous buffer must be made contiguous
• string buffers must be converted
• unevenly chunked columns must be rechunked
• data buffer must be cast from {data_dtype} to UInt32

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/269634cca801ae5e. Report an issue: GitHub.