{"record":{"id":"899a37dfae787f71","repo":"pola-rs/polars","slug":"cannot-get-buffer-length-for-buffer-with-dtype-dt","errorCode":null,"errorMessage":"cannot get buffer length for buffer with dtype {dtype!r}","messagePattern":"cannot get buffer length for buffer with dtype (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/interchange/utils.py","lineNumber":157,"sourceCode":"    elif format_str == \"tdD\":\n        return Date\n    elif format_str == \"ttu\":\n        return Time\n    elif (match := re.fullmatch(r\"tD([mun])\", format_str)) is not None:\n        time_unit = match.group(1) + \"s\"\n        return Duration(time_unit=time_unit)  # type: ignore[arg-type]\n\n    msg = f\"unsupported temporal data type: {dtype!r}\"\n    raise NotImplementedError(msg)\n\n\ndef get_buffer_length_in_elements(buffer_size: int, dtype: Dtype) -> int:\n    \"\"\"Get the length of a buffer in elements.\"\"\"\n    bits_per_element = dtype[1]\n    bytes_per_element, rest = divmod(bits_per_element, 8)\n    if rest > 0:\n        msg = f\"cannot get buffer length for buffer with dtype {dtype!r}\"\n        raise ValueError(msg)\n    return buffer_size // bytes_per_element\n\n\ndef polars_dtype_to_data_buffer_dtype(dtype: PolarsDataType) -> PolarsDataType:\n    \"\"\"Get the data type of the data buffer.\"\"\"\n    if dtype.is_integer() or dtype.is_float() or dtype == Boolean:\n        return dtype\n    elif dtype.is_temporal():\n        return Int32 if dtype == Date else Int64\n    elif dtype == String:\n        return UInt8\n    elif dtype in (Enum, Categorical):\n        return UInt32\n\n    msg = f\"unsupported data type: {dtype}\"\n    raise NotImplementedError(msg)\n","sourceCodeStart":139,"sourceCodeEnd":174,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/interchange/utils.py#L139-L174","documentation":"get_buffer_length_in_elements computes an element count as buffer_size // (bit_width // 8) and rejects any dtype whose bit width is not a whole number of bytes (divmod remainder > 0) with ValueError. In practice this means bit-packed dtypes such as bit_width-1 booleans: their length cannot be derived from byte size, so callers must pass byte-multiple dtypes and derive boolean lengths from column.size() instead.","triggerScenarios":"Calling buffer-length math (directly, or via a producer path that routes a bit-packed dtype into it) with a dtype tuple whose bit_width is 1, 3, or any non-byte-multiple value.","commonSituations":"Writing a custom interchange producer or reusing polars' interchange utils; malformed dtype metadata from third-party data; boolean buffers routed through generic length code.","solutions":["Pass only byte-multiple dtypes (bit width >= 8 and divisible by 8) to this helper","Handle bit-packed dtypes separately: use column.size() for the element count instead of buffer size math","Fix the dtype tuple produced upstream so widths are valid byte multiples"],"exampleFix":"// before\nlength = get_buffer_length_in_elements(buffer.bufsize, dtype)  # dtype bit_width=1 -> ValueError\n\n// after\nif dtype[1] % 8 != 0:\n    length = column.size()  # bit-packed: derive from column metadata\nelse:\n    length = get_buffer_length_in_elements(buffer.bufsize, dtype)","handlingStrategy":"validation","validationCode":"def buffer_length_is_derivable(dtype) -> bool:\n    bit_width = dtype[1]\n    return bit_width >= 8 and bit_width % 8 == 0\n\n# usage: derive bit-packed lengths from column.size() instead","typeGuard":null,"tryCatchPattern":"try:\n    length = get_buffer_length_in_elements(buffer.bufsize, dtype)\nexcept ValueError:\n    length = column.size()  # bit-packed dtype: derive length from metadata","preventionTips":["Never route bit-packed dtypes (bit_width 1) through byte-size length math","Use column.size() for boolean/bit-packed element counts","Validate dtype tuples (positive, byte-multiple widths) in producer tests"],"tags":["polars","interchange-protocol","buffer","dtype","value-error","producer-bug"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}