What causes "bytemask must be converted into a bitmask" in pola-rs/polars?

pola-rs/polars · error · CopyNotAllowedError

bytemask must be converted into a bitmask

Error message

bytemask must be converted into a bitmask

What it means

USE_BYTEMASK validity stores one byte per value. Polars validity is bit-packed, so a bytemask must be read as a UInt8 Series and cast to Boolean - always a copy, regardless of null_value. Under allow_copy=False Polars raises CopyNotAllowedError before touching the buffer.

Source

Thrown at py-polars/src/polars/interchange/from_dataframe.py:326

    if null_value != 0:
        if not allow_copy:
            msg = "bitmask must be inverted"
            raise CopyNotAllowedError(msg)
        s = ~s

    return s


def _construct_validity_buffer_from_bytemask(
    buffer: Buffer,
    null_value: int,
    *,
    allow_copy: bool,
) -> Series:
    if not allow_copy:
        msg = "bytemask must be converted into a bitmask"
        raise CopyNotAllowedError(msg)

    buffer_info = (buffer.ptr, 0, buffer.bufsize)
    s = pl.Series._from_buffer(UInt8, buffer_info, owner=buffer)
    s = s.cast(Boolean)

    if null_value != 0:
        s = ~s

    return s

View on GitHub (pinned to df599052da)

Solutions

Retry with allow_copy=True
Have the producer emit a packed bitmask (USE_BITMASK) instead of a bytemask
Clean nulls upstream so no validity buffer is needed at all

Example fix

// before
df = pl.from_dataframe(producer_df, allow_copy=False)  # bytemask validity -> error

// after
try:
    df = pl.from_dataframe(producer_df, allow_copy=False)
except pl.exceptions.CopyNotAllowedError:
    df = pl.from_dataframe(producer_df, allow_copy=True)

Defensive patterns

Strategy: try-catch

Validate before calling

from polars.interchange.protocol import ColumnNullType

def columns_use_bytemask(df) -> bool:
    proto = df.__dataframe__(allow_copy=False)
    return any(
        col.describe_null()[0] == ColumnNullType.USE_BYTEMASK
        for col in proto.get_columns()
    )

Try / catch

try:
    out = pl.from_dataframe(df, allow_copy=False)
except pl.exceptions.CopyNotAllowedError:
    # bytemask must be cast to bit-packed validity; allow the copy
    out = pl.from_dataframe(df, allow_copy=True)

Prevention

Producers: emit packed bitmasks instead of bytemasks for polars consumers
Remember bytemask validity always copies - never promise zero-copy for such producers
Use one shared retry-with-copy helper for every CopyNotAllowedError in the ingest path

When it happens

Trigger: allow_copy=False conversion of any column whose producer declares USE_BYTEMASK validity in describe_null.

Common situations: Simple producer implementations that find bytemasks easier to emit; performance-sensitive zero-copy consumers; together with errors 301/306/307/312, one of several copy-forcing representation mismatches.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/c8e5618bc04c04cc. Report an issue: GitHub.