{"record":{"id":"6739b7d17c2b1708","repo":"pola-rs/polars","slug":"data-buffer-must-be-cast-from-data-dtype-to-uint","errorCode":null,"errorMessage":"data buffer must be cast from {data_dtype} to UInt32","messagePattern":"data buffer must be cast from (.+?) to UInt32","errorType":"exception","errorClass":"CopyNotAllowedError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/interchange/from_dataframe.py","lineNumber":183,"sourceCode":"    data_buffer = _construct_data_buffer(\n        *buffers[\"data\"], column.size(), offset, allow_copy=allow_copy\n    )\n    validity_buffer = _construct_validity_buffer(\n        buffers[\"validity\"], column, dtype, data_buffer, offset, allow_copy=allow_copy\n    )\n\n    # First construct a physical Series without categories\n    # to allow for sentinel values that do not fit in UInt32\n    data_dtype = data_buffer.dtype\n    out = pl.Series._from_buffers(\n        data_dtype, data=data_buffer, validity=validity_buffer\n    )\n\n    # Polars only supports UInt32 categoricals\n    if data_dtype != UInt32:\n        if not allow_copy and column.size() > 0:\n            msg = f\"data buffer must be cast from {data_dtype} to UInt32\"\n            raise CopyNotAllowedError(msg)\n\n        # TODO: Cast directly to Enum\n        # https://github.com/pola-rs/polars/issues/13409\n        out = out.cast(UInt32)\n\n    return out.cast(dtype)\n\n\ndef _construct_data_buffer(\n    buffer: Buffer,\n    dtype: Dtype,\n    length: int,\n    offset: int = 0,\n    *,\n    allow_copy: bool,\n) -> Series:\n    polars_dtype = dtype_to_polars_dtype(dtype)\n","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/interchange/from_dataframe.py#L165-L201","documentation":"Polars stores categorical dictionary indices physically as UInt32. When a producer supplies indices in another width (Int8/Int16/Int32/Int64), Polars first builds the physical Series in the original dtype - so sentinel values that do not fit UInt32 survive - and then casts to UInt32 before casting to the Enum dtype. The cast allocates, so with allow_copy=False and a non-empty column it raises CopyNotAllowedError.","triggerScenarios":"pl.from_dataframe(df, allow_copy=False) on a dictionary-encoded categorical column whose indices buffer dtype is not UInt32.","commonSituations":"Zero-copy ingest paths receiving Int32 or Int64 dictionary indices from Arrow-based or custom producers; producers that also use out-of-range sentinel values, forcing the intermediate physical Series.","solutions":["Use allow_copy=True for data containing dictionary-encoded categoricals","Have the producer emit UInt32 indices buffers","Catch CopyNotAllowedError and retry with allow_copy=True at the call site"],"exampleFix":"// before\ndf = pl.from_dataframe(producer_df, allow_copy=False)  # indices are Int32 -> error\n\n// after\ntry:\n    df = pl.from_dataframe(producer_df, allow_copy=False)\nexcept pl.exceptions.CopyNotAllowedError:\n    df = pl.from_dataframe(producer_df, allow_copy=True)","handlingStrategy":"try-catch","validationCode":"def categorical_indices_are_uint32(df) -> bool:\n    from polars.interchange.protocol import DtypeKind\n    proto = df.__dataframe__(allow_copy=False)\n    for col in proto.get_columns():\n        if col.dtype[0] == DtypeKind.CATEGORICAL and col.size() > 0:\n            _, idx_dtype = col.get_buffers()['data']\n            if idx_dtype[1] != 32 or idx_dtype[2] != 'I':\n                return False\n    return True","typeGuard":null,"tryCatchPattern":"try:\n    out = pl.from_dataframe(df, allow_copy=False)\nexcept pl.exceptions.CopyNotAllowedError:\n    # UInt32 cast for dictionary indices is unavoidable; allow the copy\n    out = pl.from_dataframe(df, allow_copy=True)","preventionTips":["Producers: emit UInt32 dictionary indices to match polars' physical layout","Never assume allow_copy=False works for dictionary-encoded categoricals from foreign producers","Reserve allow_copy=False for primitive numeric frames you control end to end"],"tags":["polars","interchange-protocol","categorical","zero-copy","memory"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}