pola-rs/polars · error · ValueError

`columns` arg should only have unique values, got

Error message

`columns` arg should only have unique values, got {columns!r}

What it means

When `columns` is a list (of names or indices), all entries must be unique. `parse_columns_arg` calls `_ensure_columns_are_unique`, which raises ValueError on duplicates because selecting the same column twice is ambiguous.

Solutions

Deduplicate while preserving order: `columns = list(dict.fromkeys(columns))`
Or use `sorted(set(columns))` when order doesn't matter (for int projections)
Validate uniqueness before the call

Example fix

// before
read_csv(f, columns=['a', 'a', 'b'])
// after
read_csv(f, columns=list(dict.fromkeys(['a', 'a', 'b'])))  # ['a', 'b']

Defensive patterns

Strategy: validation

Validate before calling

if columns and len(columns) != len(set(columns)):
    columns = list(dict.fromkeys(columns))

Type guard

def has_unique(items) -> bool:
    return len(items) == len(set(items))

Try / catch

try:
    df = pl.read_csv(f, columns=columns)
except ValueError as e:
    if 'unique values' in str(e):
        columns = list(dict.fromkeys(columns))
        df = pl.read_csv(f, columns=columns)
    else:
        raise

Prevention

Deduplicate selection lists at the point they are built
Use dict.fromkeys for order-preserving dedup
Validate user-supplied column lists before read calls

When it happens

Trigger: `read_csv(f, columns=['a','a','b'])` or `columns=[0,0,2]` — len(columns) != len(set(columns)).

Common situations: Programmatically built selection lists that append duplicates; merging multiple column selections without dedup.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/be1fe6c51975fbd0. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/io/_utils.py:84

    elif isinstance(columns, int):
        projection = [columns]
    elif is_str_sequence(columns):
        _ensure_columns_are_unique(columns)
        column_names = columns
    elif is_int_sequence(columns):
        _ensure_columns_are_unique(columns)
        projection = columns
    else:
        msg = "the `columns` argument should contain a list of all integers or all string values"
        raise TypeError(msg)

    return projection, column_names


def _ensure_columns_are_unique(columns: Sequence[str] | Sequence[int]) -> None:
    if len(columns) != len(set(columns)):
        msg = f"`columns` arg should only have unique values, got {columns!r}"
        raise ValueError(msg)


def parse_row_index_args(
    row_index_name: str | None = None,
    row_index_offset: int = 0,
) -> tuple[str, int] | None:
    """
    Parse the `row_index_name` and `row_index_offset` arguments of an I/O function.

    The Rust functions take a single tuple rather than two separate arguments.
    """
    if row_index_name is None:
        return None
    else:
        return (row_index_name, row_index_offset)


@overload

View on GitHub (pinned to fe841f959e)