pola-rs/polars · error · CopyNotAllowedError

byte-packed boolean buffer must be converted to bit-packed b

Error message

byte-packed boolean buffer must be converted to bit-packed boolean

What it means

The interchange protocol allows booleans packed one byte per value (bit width 8) or one bit per value (bit width 1). Polars stores booleans bit-packed, so a byte-packed buffer must be read as UInt8 and cast to Boolean, which allocates. With allow_copy=False and a non-empty column, Polars raises CopyNotAllowedError before doing the cast.

Source

Thrown at py-polars/src/polars/interchange/from_dataframe.py:215

    *,
    allow_copy: bool,
) -> Series:
    polars_dtype = dtype_to_polars_dtype(dtype)

    # Handle implementations that incorrectly set the data buffer dtype
    # to the column dtype
    # https://github.com/pola-rs/polars/pull/10787
    polars_dtype = polars_dtype_to_data_buffer_dtype(polars_dtype)

    buffer_info = (buffer.ptr, offset, length)

    # Handle byte-packed boolean buffer
    if polars_dtype == Boolean and dtype[1] == 8:
        if length == 0:
            return pl.Series(dtype=Boolean)
        elif not allow_copy:
            msg = "byte-packed boolean buffer must be converted to bit-packed boolean"
            raise CopyNotAllowedError(msg)
        return pl.Series._from_buffer(UInt8, buffer_info, owner=buffer).cast(Boolean)

    return pl.Series._from_buffer(polars_dtype, buffer_info, owner=buffer)


def _construct_offsets_buffer(
    buffer: Buffer,
    dtype: Dtype,
    offset: int,
    *,
    allow_copy: bool,
) -> Series:
    polars_dtype = dtype_to_polars_dtype(dtype)
    length = get_buffer_length_in_elements(buffer.bufsize, dtype) - offset

    buffer_info = (buffer.ptr, offset, length)
    s = pl.Series._from_buffer(polars_dtype, buffer_info, owner=buffer)

View on GitHub (pinned to df599052da)

Solutions

1. Retry the conversion with allow_copy=True
2. Have the producer emit bit-packed (bit_width 1) boolean buffers
3. If the type change is acceptable, represent the column as UInt8 upstream to stay zero-copy

Example fix

// before
df = pl.from_dataframe(producer_df, allow_copy=False)  # byte-packed bools -> error

// after
try:
    df = pl.from_dataframe(producer_df, allow_copy=False)
except pl.exceptions.CopyNotAllowedError:
    df = pl.from_dataframe(producer_df, allow_copy=True)

def booleans_are_bit_packed(df) -> bool:
    proto = df.__dataframe__(allow_copy=False)
    for col in proto.get_columns():
        if col.dtype[0] == 20 and col.dtype[1] == 8 and col.size() > 0:  # BOOL, 8 bits
            return False
    return True

try:
    out = pl.from_dataframe(df, allow_copy=False)
except pl.exceptions.CopyNotAllowedError:
    # byte-packed bools must be cast to bit-packed; allow the copy
    out = pl.from_dataframe(df, allow_copy=True)

When it happens

Trigger: allow_copy=False conversion of a non-empty Boolean column whose protocol dtype tuple has bit_width 8 (dtype[1] == 8).

Common situations: Simple or custom producers that find byte-per-boolean buffers easier to emit; strict zero-copy pipelines in memory-constrained services.

Related errors

• non-contiguous buffer must be made contiguous
• unevenly chunked columns must be rechunked
• string buffers must be converted
• data buffer must be cast from {data_dtype} to UInt32
• offsets buffer must be cast from {polars_dtype} to Int64

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/1ae18d18fa74ab44. Report an issue: GitHub.