pola-rs/polars · error · ValueError

more new column names are specified than there are selected

Error message

more new column names are specified than there are selected columns

What it means

When new_columns is used to rename CSV columns and schema_overrides is a dict (or was converted to one), read_csv must map dtype keys back to the current names (py-polars/src/polars/io/csv/functions.py:437-448). In the explicit-`columns` branch, if len(columns) < len(new_columns) there are more new names than selected columns and ValueError is raised before parsing.

Source

Thrown at py-polars/src/polars/io/csv/functions.py:443

        # Map list of dtypes when used together with selected columns as a dtypes dict
        # so the dtypes are applied to the correct column instead of the first x
        # columns.
        schema_overrides = dict(zip(columns, schema_overrides, strict=False))

    if new_columns and schema_overrides and isinstance(schema_overrides, dict):
        current_columns = None

        # As new column names are not available yet while parsing the CSV file, rename
        # column names in dtypes to old names (if possible) so they can be used during
        # CSV parsing.
        if columns:
            if len(columns) < len(new_columns):
                msg = (
                    "more new column names are specified than there are selected"
                    " columns"
                )
                raise ValueError(msg)

            # Get column names of requested columns.
            current_columns = columns[0 : len(new_columns)]
        elif not has_header:
            # When there are no header, column names are autogenerated (and known).

            if projection:
                if columns and len(columns) < len(new_columns):
                    msg = (
                        "more new column names are specified than there are selected"
                        " columns"
                    )
                    raise ValueError(msg)
                # Convert column indices from projection to 'column_1', 'column_2', ...
                # column names.
                current_columns = [
                    f"column_{column_idx + 1}" for column_idx in projection
                ]

View on GitHub (pinned to df599052da)

Solutions

1. Make new_columns no longer than columns (drop entries for unselected columns)
2. Rename after reading with df.rename({'a': 'x'}) instead of new_columns
3. Derive columns, new_columns, and schema_overrides from one validated config object

Example fix

# before
df = pl.read_csv("f.csv", columns=["a", "b"], new_columns=["x", "y", "z"], schema_overrides={"a": pl.Int64})
# after
df = pl.read_csv("f.csv", columns=["a", "b"], new_columns=["x", "y"], schema_overrides={"a": pl.Int64})

def check_new_columns(columns, new_columns, schema_overrides) -> None:
    if new_columns and schema_overrides is not None and columns is not None:
        if len(columns) < len(new_columns):
            raise ValueError(
                f"{len(new_columns)} new names for {len(columns)} selected columns"
            )

try:
    df = pl.read_csv(path, columns=cols, new_columns=new, schema_overrides=ov)
except ValueError as e:
    if "more new column names" in str(e):
        df = pl.read_csv(path, columns=cols, schema_overrides=ov).rename(
            dict(zip(cols, new))
        )
    else:
        raise

When it happens

Trigger: pl.read_csv(f, columns=['a','b'], new_columns=['x','y','z'], schema_overrides={'a': pl.Int64}) - two selected columns but three rename targets.

Common situations: Renaming while selecting a subset: the rename list kept entries for columns that were later removed from the selection; evolving schemas where a column was dropped but new_columns was not updated.

Related errors

• specified column names do not start with 'column_', but auto
• more dtypes overrides are specified than there are selected
• `columns` arg should only have unique values, got {columns!r
• {arg_name}="{arg}" should be a single byte character or empt
• {arg_name}="{arg}" should be a single byte character, but is

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/abe14e015e0ad8e2. Report an issue: GitHub.