{"record":{"id":"68174cec8800cdb0","repo":"pola-rs/polars","slug":"dimensions-of-columns-arg-must-match-data-dimensio","errorCode":null,"errorMessage":"dimensions of columns arg must match data dimensions","messagePattern":"dimensions of columns arg must match data dimensions","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/_utils/construction/dataframe.py","lineNumber":1186,"sourceCode":"\ndef arrow_to_pydf(\n    data: pa.Table | pa.RecordBatch,\n    schema: SchemaDefinition | None = None,\n    *,\n    schema_overrides: SchemaDict | None = None,\n    strict: bool = True,\n    rechunk: bool = True,\n) -> PyDataFrame:\n    \"\"\"Construct a PyDataFrame from an Arrow Table or RecordBatch.\"\"\"\n    column_names, schema_overrides = _unpack_schema(\n        (schema or data.schema.names), schema_overrides=schema_overrides\n    )\n    try:\n        if column_names != data.schema.names:\n            data = data.rename_columns(column_names)\n    except pa.ArrowInvalid as e:\n        msg = \"dimensions of columns arg must match data dimensions\"\n        raise ValueError(msg) from e\n\n    batches: list[pa.RecordBatch]\n    if isinstance(data, pa.RecordBatch):\n        batches = [data]\n    elif data.num_columns == 0:\n        return PyDataFrame.empty_with_height(data.num_rows)\n    else:\n        batches = data.to_batches()\n\n    # supply the arrow schema so the metadata is intact\n    pydf = PyDataFrame.from_arrow_record_batches(batches, data.schema)\n\n    if rechunk:\n        pydf = pydf.rechunk()\n\n    if schema_overrides is not None:\n        pydf = _post_apply_columns(\n            pydf,","sourceCodeStart":1168,"sourceCodeEnd":1204,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/_utils/construction/dataframe.py#L1168-L1204","documentation":"Raised by arrow_to_pydf when a Polars DataFrame is constructed from a pyarrow Table/RecordBatch together with an explicit schema whose number of entries differs from the table's column count. polars calls data.rename_columns(column_names); pyarrow rejects a name list of the wrong length with ArrowInvalid, which polars re-raises as this ValueError. It enforces the invariant that the supplied column names map 1:1 onto the incoming Arrow columns.","triggerScenarios":"pl.from_arrow(table, schema=[\"a\", \"b\"]) on a table with 3 columns; pl.DataFrame(record_batch, schema=[\"x\", \"y\", \"z\", \"w\"]) on a 2-column batch; passing a list of (name, dtype) pairs whose length != data.num_columns.","commonSituations":"The upstream Arrow producer added or removed a column after a dependency upgrade; a hardcoded schema list is reused after the data contract changed; code selects a subset of columns but still passes the full name list.","solutions":["Make len(schema) equal data.num_columns (check data.schema.names before passing).","Omit schema entirely to keep the Arrow table's own column names, then rename afterwards with df.rename(...).","Align the table first: table = table.select([...]) so the selected columns match the schema you pass.","Remember (name, dtype) pairs must also match the column count — they override existing columns, they do not select a subset."],"exampleFix":"// before\ndf = pl.from_arrow(tbl, schema=[\"a\", \"b\"])  # tbl has 3 columns\n\n// after\ndf = pl.from_arrow(tbl.select([\"a\", \"b\"]), schema=[\"a\", \"b\"])\n// or simply: df = pl.from_arrow(tbl)","handlingStrategy":"validation","validationCode":"import pyarrow as pa\n\nnames = table.schema.names if isinstance(table, pa.Table) else table.schema.names\nif schema is not None and len(schema) != table.num_columns:\n    raise ValueError(\n        f\"schema has {len(schema)} entries but arrow data has {table.num_columns} columns: {names}\"\n    )\ndf = pl.from_arrow(table, schema=schema)","typeGuard":"def schema_matches_arrow(schema: list | None, data: pa.Table | pa.RecordBatch) -> bool:\n    return schema is None or len(schema) == data.num_columns","tryCatchPattern":"try:\n    df = pl.from_arrow(table, schema=schema)\nexcept ValueError as e:\n    if \"dimensions of columns arg\" in str(e):\n        raise ValueError(f\"schema {schema} vs arrow columns {table.schema.names}\") from e\n    raise","preventionTips":["Derive the schema list from data.schema.names instead of hardcoding it.","Log table.num_columns next to len(schema) at ingestion boundaries.","Pin the upstream Arrow producer's schema with a contract test."],"tags":["arrow","schema","dataframe","column-mismatch"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}