{"record":{"id":"1e51bb4672a1b8f4","repo":"pola-rs/polars","slug":"pyarrow-is-required-for-converting-a-pandas-datafr","errorCode":null,"errorMessage":"pyarrow is required for converting a pandas dataframe to Polars, unless each of its columns is a simple numpy-backed one (e.g. 'int64', 'bool', 'float32' - not 'Int64')","messagePattern":"pyarrow is required for converting a pandas dataframe to Polars, unless each of its columns is a simple numpy-backed one \\(e\\.g\\. 'int64', 'bool', 'float32' - not 'Int64'\\)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/_utils/construction/dataframe.py","lineNumber":1129,"sourceCode":"            return PyDataFrame.empty_with_height(data.shape[0])\n\n        if all(is_simple_numpy_backed_pandas_series(data[col]) for col in data.columns):\n            # Convert via NumPy directly, no PyArrow needed.\n            return pl.DataFrame(\n                {str(col): data[col].to_numpy() for col in data.columns},\n                schema=schema,\n                strict=strict,\n                schema_overrides=schema_overrides,\n                nan_to_null=nan_to_null,\n            )._df\n\n    if not _PYARROW_AVAILABLE:\n        msg = (\n            \"pyarrow is required for converting a pandas dataframe to Polars, \"\n            \"unless each of its columns is a simple numpy-backed one \"\n            \"(e.g. 'int64', 'bool', 'float32' - not 'Int64')\"\n        )\n        raise ImportError(msg)\n    arrow_dict = {}\n    length = data.shape[0]\n\n    if convert_index:\n        for idxcol in data.index.names:\n            arrow_dict[str(idxcol)] = plc.pandas_series_to_arrow(\n                # get_level_values accepts `int | str`\n                # but `index.names` returns `Hashable`\n                data.index.get_level_values(idxcol),  # type: ignore[arg-type, unused-ignore]\n                nan_to_null=nan_to_null,\n                length=length,\n            )\n\n    for col_idx, col_data in data.items():\n        arrow_dict[str(col_idx)] = plc.pandas_series_to_arrow(\n            col_data, nan_to_null=nan_to_null, length=length\n        )\n","sourceCodeStart":1111,"sourceCodeEnd":1147,"githubUrl":"https://github.com/pola-rs/polars/blob/fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e/py-polars/src/polars/_utils/construction/dataframe.py#L1111-L1147","documentation":"Raised by pandas_to_pydf when pyarrow is not installed and the pandas frame contains columns whose backing dtypes are not simple numpy types (e.g. pandas nullable 'Int64', string dtype, categoricals). Without pyarrow, polars can only convert plain numpy-backed columns and raises ImportError otherwise.","triggerScenarios":"pl.from_pandas(df) or pl.DataFrame(df) in an environment without pyarrow, where the frame has extension dtypes such as 'Int64', 'boolean', 'string', or datetime-with-timezone columns.","commonSituations":"Fresh virtualenv/CI image missing the pyarrow extra (pip install 'polars[pandas]'), pandas code migrated to nullable dtypes ('Int64') that used to convert fine, minimal deployments trimming dependencies.","solutions":["Install pyarrow: pip install pyarrow","Cast extension dtypes to numpy equivalents before conversion, e.g. df = df.astype({'col': 'int64'})","Use pl.from_pandas(df) only after ensuring all columns are numpy-backed, or convert column-by-column to polars Series"],"exampleFix":"// before\npl.from_pandas(df)  # df has pandas 'Int64' dtype, no pyarrow installed\n// after\n# pip install pyarrow\npl.from_pandas(df)","handlingStrategy":"fallback","validationCode":"import importlib.util\nif importlib.util.find_spec('pyarrow') is None:\n    raise ImportError(\"pyarrow required for this pandas conversion; pip install pyarrow\")","typeGuard":"def pyarrow_available() -> bool:\n    import importlib.util\n    return importlib.util.find_spec('pyarrow') is not None","tryCatchPattern":"try:\n    df_pl = pl.from_pandas(df)\nexcept ImportError:\n    df_pl = pl.DataFrame({str(c): df[c].to_numpy() for c in df.columns})","preventionTips":["Install the pandas extra: pip install 'polars[pandas]'","Pin pyarrow in requirements/CI images","Avoid pandas nullable extension dtypes ('Int64', 'boolean') on the hot conversion path, or cast to numpy dtypes first"],"tags":["python","polars","pandas","pyarrow","dependency"],"backgroundTag":"missing-dependency","analyzedSha":"fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e","analyzedAt":"2026-09-18T22:14:11.667Z","contentChangedAt":"2026-09-18T22:14:11.667Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}