pola-rs/polars · error · CopyNotAllowedError

offsets buffer must be cast from {polars_dtype} to Int64

Error message

offsets buffer must be cast from {polars_dtype} to Int64

What it means

Polars string offsets are Int64. If a producer provides string offsets in another integer width (Int32 is common in Arrow-style producers), Polars reads the buffer with the producer's dtype and casts it to Int64, which allocates. Under allow_copy=False this raises CopyNotAllowedError.

Source

Thrown at py-polars/src/polars/interchange/from_dataframe.py:238

def _construct_offsets_buffer(
    buffer: Buffer,
    dtype: Dtype,
    offset: int,
    *,
    allow_copy: bool,
) -> Series:
    polars_dtype = dtype_to_polars_dtype(dtype)
    length = get_buffer_length_in_elements(buffer.bufsize, dtype) - offset

    buffer_info = (buffer.ptr, offset, length)
    s = pl.Series._from_buffer(polars_dtype, buffer_info, owner=buffer)

    # Polars only supports Int64 offsets
    if polars_dtype != Int64:
        if not allow_copy:
            msg = f"offsets buffer must be cast from {polars_dtype} to Int64"
            raise CopyNotAllowedError(msg)
        s = s.cast(Int64)

    return s


def _construct_validity_buffer(
    validity_buffer_info: tuple[Buffer, Dtype] | None,
    column: Column,
    column_dtype: PolarsDataType,
    data: Series,
    offset: int = 0,
    *,
    allow_copy: bool,
) -> Series | None:
    null_type, null_value = column.describe_null
    if null_type == ColumnNullType.NON_NULLABLE or column.null_count == 0:
        return None

View on GitHub (pinned to df599052da)

Solutions

1. Use allow_copy=True for string data
2. Have the producer widen offsets buffers to Int64
3. Catch CopyNotAllowedError and retry with allow_copy=True

Example fix

// before
df = pl.from_dataframe(producer_df, allow_copy=False)  # Int32 offsets -> error

// after
try:
    df = pl.from_dataframe(producer_df, allow_copy=False)
except pl.exceptions.CopyNotAllowedError:
    df = pl.from_dataframe(producer_df, allow_copy=True)

def string_offsets_are_int64(df) -> bool:
    from polars.interchange.protocol import DtypeKind
    proto = df.__dataframe__(allow_copy=False)
    for col in proto.get_columns():
        if col.dtype[0] == DtypeKind.STRING and col.size() > 0:
            _, off_dtype = col.get_buffers()['offsets']
            if not (off_dtype[1] == 64 and off_dtype[2] == 'l'):
                return False
    return True

try:
    out = pl.from_dataframe(df, allow_copy=False)
except pl.exceptions.CopyNotAllowedError:
    # Int64 offsets cast is unavoidable; allow the copy
    out = pl.from_dataframe(df, allow_copy=True)

When it happens

Trigger: pl.from_dataframe(df, allow_copy=False) on a String column whose offsets buffer dtype is not Int64 (e.g. Int32).

Common situations: Zero-copy consumers receiving Arrow-style Int32 offsets; producer version changes that switch offsets width; combined with error 301, string columns are the least zero-copy-friendly type in the protocol path.

Related errors

• string buffers must be converted
• non-contiguous buffer must be made contiguous
• string buffers must be converted
• unevenly chunked columns must be rechunked
• data buffer must be cast from {data_dtype} to UInt32

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/44ad7ca63e9cf750. Report an issue: GitHub.