pola-rs/polars · error · TypeError

invalid sentinel value for column of type

Error message

invalid sentinel value for column of type {column_dtype}: {null_value!r}

What it means

For USE_SENTINEL columns, Polars builds a one-element Series of the sentinel value, casts it to the column dtype when temporal, and compares it against the data. If that cast/comparison raises InvalidOperationError (sentinel cannot be represented in the column's dtype), it is re-raised as TypeError naming the column dtype and the offending sentinel value.

Solutions

Fix the producer to use a sentinel representable in the column's dtype (e.g. 255 for UInt8) or a validity bitmask.
Convert the affected column through another path (e.g. pyarrow with explicit null replacement) before importing.
Catch TypeError from from_dataframe and fall back to a producer-specific converter for that frame.

Example fix

# producer side, before
 return (ColumnNullType.USE_SENTINEL, -1)  # on a UInt8 column -> TypeError
# after
 return (ColumnNullType.USE_SENTINEL, 255)  # representable in UInt8
# or, cleanest:
 return (ColumnNullType.USE_BITMAP, 0)

Defensive patterns

Strategy: try-catch

Validate before calling

# verify the sentinel is representable before importing
def sentinel_ok(col) -> bool:
    null_type, sentinel = col.describe_null()
    if null_type != ColumnNullType.USE_SENTINEL:
        return True
    try:
        s = pl.Series([sentinel])
        if col.dtype[0] == DtypeKind.DATETIME:
            s = s.cast(dtype_to_polars_dtype(col.dtype))
        return True
    except Exception:
        return False

Try / catch

try:
    df = pl.from_dataframe(obj)
except TypeError as e:
    if "invalid sentinel value" not in str(e):
        raise
    # producer bug: fix describe_null() to return a dtype-representable sentinel,
    # or convert the column via pyarrow with explicit null handling

Prevention

Choose sentinels valid for the physical buffer dtype, not the logical column type.
Prefer bitmasks over sentinels in any producer you control.
Validate sentinel/dtype pairs in producer unit tests.

When it happens

Trigger: pl.from_dataframe(obj) where a column declares a sentinel incompatible with its physical dtype — e.g. sentinel -1 on a UInt8 column, or a string sentinel on a temporal column. The producer's describe_null() returned a value that cannot legally appear in the column.

Common situations: Producers hardcoding -1 as a universal missing-value marker regardless of unsigned/physical dtype; format specs where the sentinel was designed for the logical type, not the physical buffer type.

Related errors

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/0301671ca0cc7db3. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/interchange/from_dataframe.py:291

    elif null_type == ColumnNullType.USE_NAN:
        if not allow_copy:
            msg = "bitmask must be constructed"
            raise CopyNotAllowedError(msg)
        return data.is_not_nan()

    elif null_type == ColumnNullType.USE_SENTINEL:
        if not allow_copy:
            msg = "bitmask must be constructed"
            raise CopyNotAllowedError(msg)

        sentinel = pl.Series([null_value])
        try:
            if column_dtype.is_temporal():
                sentinel = sentinel.cast(column_dtype)
            return data != sentinel  # noqa: TRY300
        except InvalidOperationError as e:
            msg = f"invalid sentinel value for column of type {column_dtype}: {null_value!r}"
            raise TypeError(msg) from e

    else:
        msg = f"unsupported null type: {null_type!r}"
        raise NotImplementedError(msg)


def _construct_validity_buffer_from_bitmask(
    buffer: Buffer,
    null_value: int,
    length: int,
    offset: int = 0,
    *,
    allow_copy: bool,
) -> Series:
    buffer_info = (buffer.ptr, offset, length)
    s = pl.Series._from_buffer(Boolean, buffer_info, buffer)

    if null_value != 0:

View on GitHub (pinned to 9b5d73fd00)