pola-rs/polars · error · ValueError

cannot set both `with_column_names` and `new_columns`; mutua

Error message

cannot set both `with_column_names` and `new_columns`; mutually exclusive

What it means

scan_csv offers two mutually exclusive ways to control header names: with_column_names (a callable that receives the parsed header list and returns a modified one) and new_columns (a fixed list of replacement names). Setting both is ambiguous - which one wins for name-dependent features like schema_overrides? - so polars rejects the combination immediately with this ValueError.

Source

Thrown at py-polars/src/polars/io/csv/functions.py:1408

    │ u16 ┆ str  │
    ╞═════╪══════╡
    │ 1   ┆ is   │
    │ 2   ┆ hard │
    │ 3   ┆ to   │
    │ 4   ┆ read │
    └─────┴──────┘
    """
    if schema_overrides is not None and not isinstance(
        schema_overrides, (dict, Sequence)
    ):
        msg = "`schema_overrides` should be of type list or dict"
        raise TypeError(msg)

    if new_columns is not None and with_column_names is not None:
        msg = (
            "cannot set both `with_column_names` and `new_columns`; mutually exclusive"
        )
        raise ValueError(msg)

    _check_arg_is_1byte("separator", separator, can_be_empty=False)
    _check_arg_is_1byte("quote_char", quote_char, can_be_empty=True)

    if isinstance(source, (str, Path)):
        source = normalize_filepath(source, check_not_directory=False)
    elif is_path_or_str_sequence(source, allow_str=False):
        source = [
            normalize_filepath(source, check_not_directory=False) for source in source
        ]

    if not infer_schema:
        infer_schema_length = 0

    if retries is not None:
        msg = "the `retries` parameter was deprecated in 1.37.1; specify 'max_retries' in `storage_options` instead."
        issue_deprecation_warning(msg)
        storage_options = storage_options or {}

View on GitHub (pinned to df599052da)

Solutions

Keep only new_columns if the full target name list is known up front
Keep only with_column_names if names must be derived from the file's header (e.g. lowercasing, stripping whitespace)
For headerless files, prefer new_columns (or a schema) since there is no header for with_column_names to transform

Example fix

# before
pl.scan_csv('f.csv',
            with_column_names=lambda cols: [c.lower() for c in cols],
            new_columns=['a', 'b'])

# after - one mechanism only
pl.scan_csv('f.csv', new_columns=['a', 'b'])
# or
pl.scan_csv('f.csv', with_column_names=lambda cols: [c.lower() for c in cols])

Defensive patterns

Strategy: validation

Validate before calling

if new_columns is not None and with_column_names is not None:
    raise ValueError(
        'pipeline config sets both new_columns and with_column_names; pick one'
    )
lf = pl.scan_csv(path, new_columns=new_columns,
                  with_column_names=with_column_names)

Prevention

In layered wrappers, let only one layer own header naming - pass through the other option untouched
Use new_columns for fixed schemas, with_column_names only when names depend on the file's own header
Add a lint/assert in shared loader utilities that the two kwargs are never both set

When it happens

Trigger: pl.scan_csv('f.csv', with_column_names=lambda cols: [c.lower() for c in cols], new_columns=['a', 'b']); any call where both kwargs are not None.

Common situations: Layered utility functions where one layer adds with_column_names normalization and a caller also passes new_columns; copy-paste from two different doc examples into one call.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/363bfb542fa36c9c. Report an issue: GitHub.