pola-rs/polars · error · ValueError

cannot get buffer length for buffer with dtype {dtype!r}

Error message

cannot get buffer length for buffer with dtype {dtype!r}

What it means

get_buffer_length_in_elements computes an element count as buffer_size // (bit_width // 8) and rejects any dtype whose bit width is not a whole number of bytes (divmod remainder > 0) with ValueError. In practice this means bit-packed dtypes such as bit_width-1 booleans: their length cannot be derived from byte size, so callers must pass byte-multiple dtypes and derive boolean lengths from column.size() instead.

Source

Thrown at py-polars/src/polars/interchange/utils.py:157

    elif format_str == "tdD":
        return Date
    elif format_str == "ttu":
        return Time
    elif (match := re.fullmatch(r"tD([mun])", format_str)) is not None:
        time_unit = match.group(1) + "s"
        return Duration(time_unit=time_unit)  # type: ignore[arg-type]

    msg = f"unsupported temporal data type: {dtype!r}"
    raise NotImplementedError(msg)


def get_buffer_length_in_elements(buffer_size: int, dtype: Dtype) -> int:
    """Get the length of a buffer in elements."""
    bits_per_element = dtype[1]
    bytes_per_element, rest = divmod(bits_per_element, 8)
    if rest > 0:
        msg = f"cannot get buffer length for buffer with dtype {dtype!r}"
        raise ValueError(msg)
    return buffer_size // bytes_per_element


def polars_dtype_to_data_buffer_dtype(dtype: PolarsDataType) -> PolarsDataType:
    """Get the data type of the data buffer."""
    if dtype.is_integer() or dtype.is_float() or dtype == Boolean:
        return dtype
    elif dtype.is_temporal():
        return Int32 if dtype == Date else Int64
    elif dtype == String:
        return UInt8
    elif dtype in (Enum, Categorical):
        return UInt32

    msg = f"unsupported data type: {dtype}"
    raise NotImplementedError(msg)

View on GitHub (pinned to df599052da)

Solutions

Pass only byte-multiple dtypes (bit width >= 8 and divisible by 8) to this helper
Handle bit-packed dtypes separately: use column.size() for the element count instead of buffer size math
Fix the dtype tuple produced upstream so widths are valid byte multiples

Example fix

// before
length = get_buffer_length_in_elements(buffer.bufsize, dtype)  # dtype bit_width=1 -> ValueError

// after
if dtype[1] % 8 != 0:
    length = column.size()  # bit-packed: derive from column metadata
else:
    length = get_buffer_length_in_elements(buffer.bufsize, dtype)

Defensive patterns

Strategy: validation

Validate before calling

def buffer_length_is_derivable(dtype) -> bool:
    bit_width = dtype[1]
    return bit_width >= 8 and bit_width % 8 == 0

# usage: derive bit-packed lengths from column.size() instead

Try / catch

try:
    length = get_buffer_length_in_elements(buffer.bufsize, dtype)
except ValueError:
    length = column.size()  # bit-packed dtype: derive length from metadata

Prevention

Never route bit-packed dtypes (bit_width 1) through byte-size length math
Use column.size() for boolean/bit-packed element counts
Validate dtype tuples (positive, byte-multiple widths) in producer tests

When it happens

Trigger: Calling buffer-length math (directly, or via a producer path that routes a bit-packed dtype into it) with a dtype tuple whose bit_width is 1, 3, or any non-byte-multiple value.

Common situations: Writing a custom interchange producer or reusing polars' interchange utils; malformed dtype metadata from third-party data; boolean buffers routed through generic length code.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/899a37dfae787f71. Report an issue: GitHub.