{"record":{"id":"f601cc87739a3caa","repo":"pola-rs/polars","slug":"cannot-parse-python-data-type-dtype-r-into-arrow","errorCode":null,"errorMessage":"cannot parse Python data type {dtype!r} into Arrow data type","messagePattern":"cannot parse Python data type (.+?) into Arrow data type","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/datatypes/convert.py","lineNumber":279,"sourceCode":"\n\ndef dtype_to_py_type(dtype: PolarsDataType) -> PythonDataType:\n    \"\"\"Convert a Polars dtype to a Python dtype.\"\"\"\n    try:\n        dtype = dtype.base_type()\n        return DataTypeMappings.DTYPE_TO_PY_TYPE[dtype]\n    except KeyError:  # pragma: no cover\n        msg = f\"conversion of polars data type {dtype!r} to Python type not implemented\"\n        raise NotImplementedError(msg) from None\n\n\ndef py_type_to_arrow_type(dtype: PythonDataType) -> pa.DataType:\n    \"\"\"Convert a Python dtype to an Arrow dtype.\"\"\"\n    try:\n        return DataTypeMappings.PY_TYPE_TO_ARROW_TYPE[dtype]\n    except KeyError:  # pragma: no cover\n        msg = f\"cannot parse Python data type {dtype!r} into Arrow data type\"\n        raise ValueError(msg) from None\n\n\ndef dtype_short_repr_to_dtype(dtype_string: str | None) -> PolarsDataType | None:\n    \"\"\"Map a PolarsDataType short repr (eg: 'i64', 'list[str]') back into a dtype.\"\"\"\n    if dtype_string is None:\n        return None\n\n    m = re.match(r\"^(\\w+)(?:\\[(.+)\\])?$\", dtype_string)\n    if m is None:\n        return None\n\n    dtype_base, subtype = m.groups()\n    dtype = DataTypeMappings.REPR_TO_DTYPE.get(dtype_base)\n    if dtype and subtype:\n        # TODO: further-improve handling for nested types (such as List,Struct)\n        try:\n            if dtype == Decimal:\n                subtype = (None, int(subtype))","sourceCodeStart":261,"sourceCodeEnd":297,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/datatypes/convert.py#L261-L297","documentation":"py_type_to_arrow_type maps a Python type to a pyarrow DataType through a fixed dictionary. A Python type outside that dictionary — complex, custom classes, numpy scalar types instead of Python types — raises ValueError. It means schema inference was handed a Python type polars cannot express in Arrow.","triggerScenarios":"Calling polars.datatypes.convert.py_type_to_arrow_type with unsupported types: complex, user-defined classes, np.int64/np.float32 used as type objects, or other types missing from PY_TYPE_TO_ARROW_TYPE.","commonSituations":"Building a pyarrow schema from type hints that include complex numbers or custom classes; passing numpy scalar types where Python types were intended; generic schema-inference utilities.","solutions":["Pass plain Python types (int, float, str, bool, bytes, datetime.datetime, datetime.date, datetime.time, datetime.timedelta)","Convert numpy scalar types to Python types first (np.int64 -> int)","For custom classes, map them explicitly to a supported type (usually str) before schema creation"],"exampleFix":"# before\narrow_t = py_type_to_arrow_type(complex)  # ValueError\n\n# after\nclass Money:\n    ...\narrow_t = py_type_to_arrow_type(str)  # serialise Money as its string form","handlingStrategy":"type-guard","validationCode":"import datetime as dt\n\nSUPPORTED_PY_TYPES = {bool, int, float, str, bytes, dt.date, dt.datetime, dt.time, dt.timedelta}\n\nif py_type not in SUPPORTED_PY_TYPES:\n    raise ValueError(f'unsupported Python type for Arrow conversion: {py_type!r}')\narrow_t = py_type_to_arrow_type(py_type)","typeGuard":"import datetime as dt\n\ndef is_arrow_mappable_py_type(t: type) -> bool:\n    return t in {bool, int, float, str, bytes, dt.date, dt.datetime, dt.time, dt.timedelta}","tryCatchPattern":"from polars.datatypes.convert import py_type_to_arrow_type\n\ntry:\n    arrow_t = py_type_to_arrow_type(py_type)\nexcept ValueError:\n    arrow_t = pa.string()  # degrade custom/exotic types to string","preventionTips":["Normalise numpy scalar types (np.int64 -> int) before schema building","Map custom classes explicitly to a supported primitive instead of passing them through"],"tags":["arrow","dtype","type-mapping","schema"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}