{"record":{"id":"be1fe6c51975fbd0","repo":"pola-rs/polars","slug":"columns-arg-should-only-have-unique-values-got","errorCode":null,"errorMessage":"`columns` arg should only have unique values, got {columns!r}","messagePattern":"`columns` arg should only have unique values, got (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/_utils.py","lineNumber":84,"sourceCode":"    elif isinstance(columns, int):\n        projection = [columns]\n    elif is_str_sequence(columns):\n        _ensure_columns_are_unique(columns)\n        column_names = columns\n    elif is_int_sequence(columns):\n        _ensure_columns_are_unique(columns)\n        projection = columns\n    else:\n        msg = \"the `columns` argument should contain a list of all integers or all string values\"\n        raise TypeError(msg)\n\n    return projection, column_names\n\n\ndef _ensure_columns_are_unique(columns: Sequence[str] | Sequence[int]) -> None:\n    if len(columns) != len(set(columns)):\n        msg = f\"`columns` arg should only have unique values, got {columns!r}\"\n        raise ValueError(msg)\n\n\ndef parse_row_index_args(\n    row_index_name: str | None = None,\n    row_index_offset: int = 0,\n) -> tuple[str, int] | None:\n    \"\"\"\n    Parse the `row_index_name` and `row_index_offset` arguments of an I/O function.\n\n    The Rust functions take a single tuple rather than two separate arguments.\n    \"\"\"\n    if row_index_name is None:\n        return None\n    else:\n        return (row_index_name, row_index_offset)\n\n\n@overload","sourceCodeStart":66,"sourceCodeEnd":102,"githubUrl":"https://github.com/pola-rs/polars/blob/fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e/py-polars/src/polars/io/_utils.py#L66-L102","documentation":"When `columns` is a list (of names or indices), all entries must be unique. `parse_columns_arg` calls `_ensure_columns_are_unique`, which raises ValueError on duplicates because selecting the same column twice is ambiguous.","triggerScenarios":"`read_csv(f, columns=['a','a','b'])` or `columns=[0,0,2]` — len(columns) != len(set(columns)).","commonSituations":"Programmatically built selection lists that append duplicates; merging multiple column selections without dedup.","solutions":["Deduplicate while preserving order: `columns = list(dict.fromkeys(columns))`","Or use `sorted(set(columns))` when order doesn't matter (for int projections)","Validate uniqueness before the call"],"exampleFix":"// before\nread_csv(f, columns=['a', 'a', 'b'])\n// after\nread_csv(f, columns=list(dict.fromkeys(['a', 'a', 'b'])))  # ['a', 'b']","handlingStrategy":"validation","validationCode":"if columns and len(columns) != len(set(columns)):\n    columns = list(dict.fromkeys(columns))","typeGuard":"def has_unique(items) -> bool:\n    return len(items) == len(set(items))","tryCatchPattern":"try:\n    df = pl.read_csv(f, columns=columns)\nexcept ValueError as e:\n    if 'unique values' in str(e):\n        columns = list(dict.fromkeys(columns))\n        df = pl.read_csv(f, columns=columns)\n    else:\n        raise","preventionTips":["Deduplicate selection lists at the point they are built","Use dict.fromkeys for order-preserving dedup","Validate user-supplied column lists before read calls"],"tags":["python","polars","io","value-error"],"backgroundTag":"invalid-argument-value","analyzedSha":"fe841f959ef4d2ceefc05a310d33ed7b1ab24e5e","analyzedAt":"2026-09-18T22:14:11.667Z","contentChangedAt":"2026-09-18T22:14:11.667Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}