pola-rs/polars · error · ValueError

more dtypes overrides are specified than there are selected

Error message

more dtypes overrides are specified than there are selected columns

What it means

The sibling check for string columns (py-polars/src/polars/io/csv/functions.py:423-433): when columns=['a','b'] and schema_overrides is a list, polars zips them into a name-to-dtype dict. If len(columns) < len(schema_overrides), the zip would silently drop the extra dtypes, so ValueError ('more dtypes overrides are specified than there are selected columns') is raised instead.

Source

Thrown at py-polars/src/polars/io/csv/functions.py:424

    if projection and schema_overrides and isinstance(schema_overrides, list):
        if len(projection) < len(schema_overrides):
            msg = "more schema overrides are specified than there are selected columns"
            raise ValueError(msg)

        # Fix list of dtypes when used together with projection as polars CSV reader
        # wants a list of dtypes for the x first columns before it does the projection.
        dtypes_list: list[PolarsDataType] = [String] * (max(projection) + 1)

        for idx, column_idx in enumerate(projection):
            if idx < len(schema_overrides):
                dtypes_list[column_idx] = schema_overrides[idx]

        schema_overrides = dtypes_list

    if columns and schema_overrides and isinstance(schema_overrides, list):
        if len(columns) < len(schema_overrides):
            msg = "more dtypes overrides are specified than there are selected columns"
            raise ValueError(msg)

        # Map list of dtypes when used together with selected columns as a dtypes dict
        # so the dtypes are applied to the correct column instead of the first x
        # columns.
        schema_overrides = dict(zip(columns, schema_overrides, strict=False))

    if new_columns and schema_overrides and isinstance(schema_overrides, dict):
        current_columns = None

        # As new column names are not available yet while parsing the CSV file, rename
        # column names in dtypes to old names (if possible) so they can be used during
        # CSV parsing.
        if columns:
            if len(columns) < len(new_columns):
                msg = (
                    "more new column names are specified than there are selected"
                    " columns"
                )

View on GitHub (pinned to df599052da)

Solutions

  1. Match lengths: exactly one dtype per selected column
  2. Use a dict {'a': pl.Int64} for sparse, name-keyed overrides
  3. Generate columns and schema_overrides from the same single config source

Example fix

# before
df = pl.read_csv("f.csv", columns=["a", "b"], schema_overrides=[pl.Int64, pl.Utf8, pl.Date])
# after
df = pl.read_csv("f.csv", columns=["a", "b"], schema_overrides={"a": pl.Int64})
Defensive patterns

Strategy: validation

Validate before calling

def check_columns_overrides(columns, schema_overrides) -> None:
    if (
        isinstance(columns, list)
        and isinstance(schema_overrides, list)
        and len(columns) < len(schema_overrides)
    ):
        raise ValueError(
            f"{len(schema_overrides)} dtypes for {len(columns)} named columns; trim or use a dict"
        )

Try / catch

try:
    df = pl.read_csv(path, columns=cols, schema_overrides=ov)
except ValueError as e:
    if "more dtypes overrides" in str(e) and isinstance(ov, list):
        df = pl.read_csv(path, columns=cols, schema_overrides=dict(zip(cols, ov)))
    else:
        raise

Prevention

When it happens

Trigger: pl.read_csv(f, columns=['a','b'], schema_overrides=[pl.Int64, pl.Utf8, pl.Date]) - two names, three dtypes.

Common situations: Dtype lists inherited from a wider schema than the currently selected columns; merging overrides from multiple configs without re-trimming to the selection.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/f530b0847d5618dd. Report an issue: GitHub.