pola-rs/polars · error · CopyNotAllowedError

offsets buffer must be cast from

Error message

offsets buffer must be cast from {polars_dtype} to Int64

What it means

Polars string offsets are Int64. When an imported string column's offsets buffer uses another width (Int32 is the common Arrow/interchange layout, Int16/Int32 from other producers), _construct_offsets_buffer must cast the offsets Series to Int64, which allocates; with allow_copy=False this raises CopyNotAllowedError.

Solutions

1. Call pl.from_dataframe(obj, allow_copy=True) to permit the Int64 cast.
2. Have the producer emit Int64 offsets (e.g. Arrow large_utf8-style layout).
3. Catch CopyNotAllowedError and retry with allow_copy=True.

Example fix

# before
df = pl.from_dataframe(obj, allow_copy=False)  # int32 offsets -> CopyNotAllowedError

# after
df = pl.from_dataframe(obj, allow_copy=True)   # offsets cast to Int64

def needs_offsets_cast(icdf) -> bool:
    for i in range(icdf.num_columns()):
        col = icdf.get_column(i)
        if col.dtype[0] != DtypeKind.STRING:
            continue
        offsets = col.get_buffers()["offsets"]
        if offsets is not None and offsets[1][:2] != (DtypeKind.INT, 64):
            return True
    return False

allow_copy = not needs_offsets_cast(obj.__dataframe__())

from polars.interchange.protocol import CopyNotAllowedError

try:
    df = pl.from_dataframe(obj, allow_copy=False)
except CopyNotAllowedError:
    df = pl.from_dataframe(obj, allow_copy=True)  # allows the Int64 offsets cast

When it happens

Trigger: pl.from_dataframe(obj, allow_copy=False) over any string (or string-categorical) column whose offsets buffer dtype is not Int64 — e.g. Arrow string arrays that use int32 offsets by default.

Common situations: Zero-copy ingestion of text data from Arrow-based producers; assuming string columns can be imported copy-free; large-string vs string layout differences between producers.

Related errors

• string buffers must be converted
• bitmask must be constructed
• bitmask must be inverted
• byte-packed boolean buffer must be converted to bit-packed…
• bytemask must be converted into a bitmask

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/44ad7ca63e9cf750. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/interchange/from_dataframe.py:238

def _construct_offsets_buffer(
    buffer: Buffer,
    dtype: Dtype,
    offset: int,
    *,
    allow_copy: bool,
) -> Series:
    polars_dtype = dtype_to_polars_dtype(dtype)
    length = get_buffer_length_in_elements(buffer.bufsize, dtype) - offset

    buffer_info = (buffer.ptr, offset, length)
    s = pl.Series._from_buffer(polars_dtype, buffer_info, owner=buffer)

    # Polars only supports Int64 offsets
    if polars_dtype != Int64:
        if not allow_copy:
            msg = f"offsets buffer must be cast from {polars_dtype} to Int64"
            raise CopyNotAllowedError(msg)
        s = s.cast(Int64)

    return s


def _construct_validity_buffer(
    validity_buffer_info: tuple[Buffer, Dtype] | None,
    column: Column,
    column_dtype: PolarsDataType,
    data: Series,
    offset: int = 0,
    *,
    allow_copy: bool,
) -> Series | None:
    null_type, null_value = column.describe_null
    if null_type == ColumnNullType.NON_NULLABLE or column.null_count == 0:
        return None

View on GitHub (pinned to 9b5d73fd00)