pola-rs/polars · error · CopyNotAllowedError

bitmask must be inverted

Error message

bitmask must be inverted

What it means

For USE_BITMAP validity buffers, Polars assumes the standard semantics bit=1 means valid, null_value=0. If the producer reports null_value != 0 (inverted bitmask, where set bits mark nulls), Polars must invert the mask with ~s — an allocation — so with allow_copy=False it raises CopyNotAllowedError('bitmask must be inverted').

Solutions

1. Call pl.from_dataframe(obj, allow_copy=True) to permit the inversion.
2. Have the producer emit a standard validity bitmask with null_value 0.
3. Catch CopyNotAllowedError and retry with allow_copy=True.

Example fix

# before
df = pl.from_dataframe(obj, allow_copy=False)  # inverted bitmask -> CopyNotAllowedError

# after
df = pl.from_dataframe(obj, allow_copy=True)   # ~mask computed

def has_inverted_bitmask(icdf) -> bool:
    return any(
        icdf.get_column(i).describe_null() == (ColumnNullType.USE_BITMAP, 1)
        for i in range(icdf.num_columns())
    )

allow_copy = not has_inverted_bitmask(obj.__dataframe__())

from polars.interchange.protocol import CopyNotAllowedError

try:
    df = pl.from_dataframe(obj, allow_copy=False)
except CopyNotAllowedError:
    df = pl.from_dataframe(obj, allow_copy=True)  # allows ~mask inversion

When it happens

Trigger: pl.from_dataframe(obj, allow_copy=False) where a column's describe_null() == (ColumnNullType.USE_BITMAP, 1) — producers that store null flags instead of validity flags.

Common situations: Interchange producers derived from numpy boolean masks (True = missing) that expose them as inverted bitmasks; zero-copy pipelines expecting standard Arrow-style validity bitmaps.

Related errors

• bitmask must be constructed
• bytemask must be converted into a bitmask
• byte-packed boolean buffer must be converted to bit-packed…
• data buffer must be cast from
• offsets buffer must be cast from

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/ba422490df2577b4. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/interchange/from_dataframe.py:312

        msg = f"unsupported null type: {null_type!r}"
        raise NotImplementedError(msg)


def _construct_validity_buffer_from_bitmask(
    buffer: Buffer,
    null_value: int,
    length: int,
    offset: int = 0,
    *,
    allow_copy: bool,
) -> Series:
    buffer_info = (buffer.ptr, offset, length)
    s = pl.Series._from_buffer(Boolean, buffer_info, buffer)

    if null_value != 0:
        if not allow_copy:
            msg = "bitmask must be inverted"
            raise CopyNotAllowedError(msg)
        s = ~s

    return s


def _construct_validity_buffer_from_bytemask(
    buffer: Buffer,
    null_value: int,
    *,
    allow_copy: bool,
) -> Series:
    if not allow_copy:
        msg = "bytemask must be converted into a bitmask"
        raise CopyNotAllowedError(msg)

    buffer_info = (buffer.ptr, 0, buffer.bufsize)
    s = pl.Series._from_buffer(UInt8, buffer_info, owner=buffer)
    s = s.cast(Boolean)

View on GitHub (pinned to 9b5d73fd00)