pola-rs/polars · error · ValueError

more schema overrides are specified than there are selected

Error message

more schema overrides are specified than there are selected columns

What it means

When read_csv receives an integer projection (columns=[0,2]) plus a list of schema_overrides, it must expand the override list to cover positions before the projection (py-polars/src/polars/io/csv/functions.py:408-419). If len(projection) < len(schema_overrides), there are more dtypes than selected columns and no valid mapping exists, so ValueError is raised before parsing.

Source

Thrown at py-polars/src/polars/io/csv/functions.py:409

            # to 'column_1', 'column_2', ...
            tbl = tbl.rename_columns(
                [f"column_{int(column[1:]) + 1}" for column in tbl.column_names]
            )
        elif not columns and projection:
            # User selected columns by positional index (e.g. `columns=[0, 2]`).
            # pyarrow's include_columns only accepts names, so the read above
            # fetched every column; pick out the requested positions now.
            tbl = tbl.select(list(projection))

        df = pl.DataFrame._from_arrow(tbl, rechunk=rechunk)
        if new_columns:
            return _update_columns(df, new_columns)
        return df

    if projection and schema_overrides and isinstance(schema_overrides, list):
        if len(projection) < len(schema_overrides):
            msg = "more schema overrides are specified than there are selected columns"
            raise ValueError(msg)

        # Fix list of dtypes when used together with projection as polars CSV reader
        # wants a list of dtypes for the x first columns before it does the projection.
        dtypes_list: list[PolarsDataType] = [String] * (max(projection) + 1)

        for idx, column_idx in enumerate(projection):
            if idx < len(schema_overrides):
                dtypes_list[column_idx] = schema_overrides[idx]

        schema_overrides = dtypes_list

    if columns and schema_overrides and isinstance(schema_overrides, list):
        if len(columns) < len(schema_overrides):
            msg = "more dtypes overrides are specified than there are selected columns"
            raise ValueError(msg)

        # Map list of dtypes when used together with selected columns as a dtypes dict
        # so the dtypes are applied to the correct column instead of the first x

View on GitHub (pinned to df599052da)

Solutions

1. Trim the dtype list so there is exactly one dtype per projected column, in projection order
2. Switch to a dict keyed by column names so the mapping is unambiguous
3. Drop columns= and .select() the needed columns after reading

Example fix

# before
df = pl.read_csv("f.csv", columns=[0], schema_overrides=[pl.Int64, pl.Int64])
# after
df = pl.read_csv("f.csv", columns=[0], schema_overrides=[pl.Int64])

def check_projection_overrides(columns, schema_overrides) -> None:
    if (
        isinstance(columns, list)
        and columns
        and all(isinstance(c, int) for c in columns)
        and isinstance(schema_overrides, list)
        and len(columns) < len(schema_overrides)
    ):
        raise ValueError(
            f"{len(schema_overrides)} dtypes for {len(columns)} projected columns; trim the list"
        )

try:
    df = pl.read_csv(path, columns=cols, schema_overrides=ov)
except ValueError as e:
    if "more schema overrides" in str(e) and isinstance(ov, list):
        df = pl.read_csv(path, columns=cols, schema_overrides=ov[: len(cols)])
    else:
        raise

When it happens

Trigger: pl.read_csv(f, columns=[0], schema_overrides=[pl.Int64, pl.Int64]) - one projected column but two dtypes. Dict overrides don't hit this check (they're keyed by name).

Common situations: Reusing a full-file dtype list after adding a columns projection as an optimization; config-driven dtype lists drifting out of sync with column selections during schema evolution.

Related errors

• more dtypes overrides are specified than there are selected
• cannot use glob patterns and integer based projection as `co
• {arg_name}="{arg}" should be a single byte character or empt
• {arg_name}="{arg}" should be a single byte character, but is
• specified column names do not start with 'column_', but auto

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/89cec8a2d8844e24. Report an issue: GitHub.