pola-rs/polars · error · TypeError

expected PyArrow Table, Array, or one or more RecordBatches;

Error message

expected PyArrow Table, Array, or one or more RecordBatches; got {qualified_type_name(data)!r}

What it means

`pl.from_arrow(data)` accepts only `pyarrow.Table`, `pyarrow.Array`/`ChunkedArray`, `pa.RecordBatch`, or a sequence of RecordBatches. Anything else — a `pa.Dataset`, an already-polars DataFrame, a pandas object, an arrow scalar — falls through to a TypeError that names the qualified type of what you passed.

Source

Thrown at py-polars/src/polars/convert/general.py:607

        )

    if isinstance(data, Iterable):
        pa_table = pa.Table.from_batches(
            itertools.chain.from_iterable(
                (b.to_batches() if isinstance(b, pa.Table) else [b]) for b in data
            )
        )
        return wrap_df(
            arrow_to_pydf(
                data=pa_table,
                rechunk=rechunk,
                schema=schema,
                schema_overrides=schema_overrides,
            )
        )

    msg = f"expected PyArrow Table, Array, or one or more RecordBatches; got {qualified_type_name(data)!r}"
    raise TypeError(msg)


@overload
def from_pandas(
    data: pd.DataFrame,
    *,
    schema_overrides: SchemaDict | None = ...,
    rechunk: bool = ...,
    nan_to_null: bool = ...,
    include_index: bool = ...,
) -> DataFrame: ...


@overload
def from_pandas(
    data: pd.Series[Any] | pd.Index[Any] | pd.DatetimeIndex,
    *,
    schema_overrides: SchemaDict | None = ...,

View on GitHub (pinned to 5d8ebabf11)

Solutions

1. For a `pa.Dataset`, materialize first: `pl.from_arrow(ds.to_table())` — or better, scan the files directly with `pl.scan_parquet`/`pl.scan_ipc`.
2. If the input is already a polars DataFrame/Series, skip the conversion entirely.
3. Type-check before calling: Table, Array, ChunkedArray, RecordBatch, or a list of batches.

Example fix

# before
ds = pa.dataset("s3://bucket/data/")
df = pl.from_arrow(ds)  # TypeError: expected PyArrow Table, Array, ...

# after
df = pl.scan_parquet("s3://bucket/data/").collect()  # or pl.from_arrow(ds.to_table())

import pyarrow as pa

def arrow_to_df(data):
    if isinstance(data, pa.Dataset):
        data = data.to_table()
    return pl.from_arrow(data)

import pyarrow as pa
from typing import Any, TypeGuard

ArrowInput = pa.Table | pa.Array | pa.ChunkedArray | pa.RecordBatch | list[pa.RecordBatch] | tuple[pa.RecordBatch, ...]

def is_arrow_convertible(data: Any) -> TypeGuard[ArrowInput]:
    return isinstance(data, (pa.Table, pa.Array, pa.ChunkedArray, pa.RecordBatch)) or (
        isinstance(data, (list, tuple)) and bool(data) and all(isinstance(b, pa.RecordBatch) for b in data)
    )

When it happens

Trigger: `pl.from_arrow(pa.dataset('data_dir'))` (a Dataset — call `.to_table()` first); `pl.from_arrow(pl.DataFrame(...))` (already polars); `pl.from_arrow(pa.scalar(1))`; `pl.from_arrow(pa.feather.read_table(...))` is fine but a flight `FlightStream` is not.

Common situations: Assuming any pyarrow object converts; double-converting data that is already a polars DataFrame; passing datasets or IPC readers instead of materialized tables.

Related errors

• expected pandas DataFrame or Series, got {qualified_type_nam
• expected data of type Sequence, got {type(data).__name__!r}
• activate 'dtype-array'
• Deserialization from JSON not implemented for {adt:?}
• `df` of type {qualified_type_name(df)!r} does not support th

AI-assisted analysis of pola-rs/polars@5d8ebabf11 (2026-08-19). Data as JSON: /api/errors/39f7f350a521086e. Report an issue: GitHub.