pola-rs/polars · error · TypeError

cannot convert List column {nm!r} to {target} (use Array dty

Error message

cannot convert List column {nm!r} to {target} (use Array dtype instead)

What it means

frame_to_numpy (backing DataFrame.to_torch and DataFrame.to_jax) must produce a rectangular, fixed-stride array. A variable-length List(pl.List) column has no fixed shape, so conversion fails with TypeError telling you to use the fixed-shape Array dtype (pl.Array(inner, width)) instead.

Source

Thrown at py-polars/src/polars/ml/utilities.py:20

from polars import DataFrame
from polars._dependencies import numpy as np
from polars._typing import IndexOrder
from polars.datatypes import Array, List


def frame_to_numpy(
    df: DataFrame,
    *,
    writable: bool,
    target: str,
    order: IndexOrder = "fortran",
) -> np.ndarray[Any, Any]:
    """Convert a DataFrame to a NumPy array for use with Jax or PyTorch."""
    for nm, tp in df.schema.items():
        if tp == List:
            msg = f"cannot convert List column {nm!r} to {target} (use Array dtype instead)"
            raise TypeError(msg) from None

    if df.width == 1 and df.schema.dtypes()[0] == Array:
        arr = df[df.columns[0]].to_numpy(writable=writable)
    else:
        arr = df.to_numpy(writable=writable, order=order)

    if arr.dtype == object:
        msg = f"cannot convert DataFrame to {target} (mixed type columns result in `object` dtype)\n{df.schema!r}"
        raise TypeError(msg)
    return arr

View on GitHub (pinned to df599052da)

Solutions

Pad to a fixed width and cast: pl.col('x').list.eval(...).cast(pl.Array(inner, width)) or df.cast({'x': pl.Array(pl.Int64, 3)})
Drop or explode the list column before conversion if it is not a feature
Convert the list column separately (e.g. row-by-row tensors) and keep the rectangular frame for the rest

Example fix

# before
pl.DataFrame({'x': [[1, 2], [3]]}).to_torch()  # List dtype -> TypeError

# after
pl.DataFrame({'x': [[1, 2], [3]]}).with_columns(
    pl.col('x').list.pad_end(3).cast(pl.Array(pl.Int64, 3))
).to_torch()

Defensive patterns

Strategy: validation

Validate before calling

list_cols = [name for name, tp in df.schema.items() if tp == pl.List]
if list_cols:
    raise TypeError(f'List columns cannot become tensors: {list_cols}; cast to pl.Array first')

Type guard

import polars as pl

def is_tensor_convertible(df: pl.DataFrame) -> bool:
    return all(tp != pl.List for tp in df.schema.values())

Prevention

Make ragged columns fixed-width (list.pad_end + cast to pl.Array) before to_torch/to_jax
Check df.schema for pl.List before ML export in data-validation steps

When it happens

Trigger: df.to_torch() or df.to_jax(return_type='array') on a frame containing a pl.List column; list columns produced by agg(...implode()), list literals, or str.split.

Common situations: ML feature frames with ragged per-row sequences (tokens, sensors, histories) fed straight into to_torch/to_jax without padding.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/1c3c9ab5cafd00a0. Report an issue: GitHub.