dimensions of columns arg must match data dimensions

Error message

dimensions of columns arg must match data dimensions

What it means

Raised by arrow_to_pydf when a Polars DataFrame is constructed from a pyarrow Table/RecordBatch together with an explicit schema whose number of entries differs from the table's column count. polars calls data.rename_columns(column_names); pyarrow rejects a name list of the wrong length with ArrowInvalid, which polars re-raises as this ValueError. It enforces the invariant that the supplied column names map 1:1 onto the incoming Arrow columns.

Source

Thrown at py-polars/src/polars/_utils/construction/dataframe.py:1186

def arrow_to_pydf(
    data: pa.Table | pa.RecordBatch,
    schema: SchemaDefinition | None = None,
    *,
    schema_overrides: SchemaDict | None = None,
    strict: bool = True,
    rechunk: bool = True,
) -> PyDataFrame:
    """Construct a PyDataFrame from an Arrow Table or RecordBatch."""
    column_names, schema_overrides = _unpack_schema(
        (schema or data.schema.names), schema_overrides=schema_overrides
    )
    try:
        if column_names != data.schema.names:
            data = data.rename_columns(column_names)
    except pa.ArrowInvalid as e:
        msg = "dimensions of columns arg must match data dimensions"
        raise ValueError(msg) from e

    batches: list[pa.RecordBatch]
    if isinstance(data, pa.RecordBatch):
        batches = [data]
    elif data.num_columns == 0:
        return PyDataFrame.empty_with_height(data.num_rows)
    else:
        batches = data.to_batches()

    # supply the arrow schema so the metadata is intact
    pydf = PyDataFrame.from_arrow_record_batches(batches, data.schema)

    if rechunk:
        pydf = pydf.rechunk()

    if schema_overrides is not None:
        pydf = _post_apply_columns(
            pydf,

View on GitHub (pinned to df599052da)

Solutions

Make len(schema) equal data.num_columns (check data.schema.names before passing).
Omit schema entirely to keep the Arrow table's own column names, then rename afterwards with df.rename(...).
Align the table first: table = table.select([...]) so the selected columns match the schema you pass.
Remember (name, dtype) pairs must also match the column count — they override existing columns, they do not select a subset.

Example fix

// before
df = pl.from_arrow(tbl, schema=["a", "b"])  # tbl has 3 columns

// after
df = pl.from_arrow(tbl.select(["a", "b"]), schema=["a", "b"])
// or simply: df = pl.from_arrow(tbl)

Defensive patterns

Strategy: validation

Validate before calling

import pyarrow as pa

names = table.schema.names if isinstance(table, pa.Table) else table.schema.names
if schema is not None and len(schema) != table.num_columns:
    raise ValueError(
        f"schema has {len(schema)} entries but arrow data has {table.num_columns} columns: {names}"
    )
df = pl.from_arrow(table, schema=schema)

Type guard

def schema_matches_arrow(schema: list | None, data: pa.Table | pa.RecordBatch) -> bool:
    return schema is None or len(schema) == data.num_columns

Try / catch

try:
    df = pl.from_arrow(table, schema=schema)
except ValueError as e:
    if "dimensions of columns arg" in str(e):
        raise ValueError(f"schema {schema} vs arrow columns {table.schema.names}") from e
    raise

Prevention

Derive the schema list from data.schema.names instead of hardcoding it.
Log table.num_columns next to len(schema) at ingestion boundaries.
Pin the upstream Arrow producer's schema with a contract test.

When it happens

Trigger: pl.from_arrow(table, schema=["a", "b"]) on a table with 3 columns; pl.DataFrame(record_batch, schema=["x", "y", "z", "w"]) on a 2-column batch; passing a list of (name, dtype) pairs whose length != data.num_columns.

Common situations: The upstream Arrow producer added or removed a column after a dependency upgrade; a hardcoded schema list is reused after the data contract changed; code selects a subset of columns but still passes the full name list.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/68174cec8800cdb0. Report an issue: GitHub.