duplicate column names found: {values.columns.tolist()}

Error message

duplicate column names found: {values.columns.tolist()}

What it means

During pandas-to-polars conversion, polars calls get_first_non_none on what it assumes is a Series. When the source pandas DataFrame has duplicated column labels, selecting that label (df["a"]) returns a 2-D DataFrame, not a Series; polars detects the stray .columns attribute and refuses rather than silently converting ambiguous data.

Source

Thrown at py-polars/src/polars/_utils/construction/other.py:49

    Returns
    -------
    :class:`pyarrow.Array`
    """
    dtype = getattr(values, "dtype", None)
    if dtype == "object":
        first_non_none = get_first_non_none(values.values)  # type: ignore[arg-type]
        if isinstance(first_non_none, str):
            return pa.array(values, pa.large_utf8(), from_pandas=nan_to_null)
        elif first_non_none is None:
            return pa.nulls(length or len(values), pa.large_utf8())
        return pa.array(values, from_pandas=nan_to_null)
    elif dtype:
        return pa.array(values, from_pandas=nan_to_null)
    else:
        # Pandas Series is actually a Pandas DataFrame when the original DataFrame
        # contains duplicated columns and a duplicated column is requested with df["a"].
        msg = "duplicate column names found: "
        raise ValueError(
            msg,
            f"{values.columns.tolist()!s}",  # type: ignore[union-attr]
        )


def coerce_arrow(array: pa.Array) -> pa.Array:
    """..."""
    import pyarrow.compute as pc

    if hasattr(array, "num_chunks") and array.num_chunks > 1:
        # small integer keys can often not be combined, so let's already cast
        # to the uint32 used by polars
        if pa.types.is_dictionary(array.type) and (
            pa.types.is_int8(array.type.index_type)
            or pa.types.is_uint8(array.type.index_type)
            or pa.types.is_int16(array.type.index_type)
            or pa.types.is_uint16(array.type.index_type)
            or pa.types.is_int32(array.type.index_type)

View on GitHub (pinned to df599052da)

Solutions

Deduplicate labels before selecting: df = df.loc[:, ~df.columns.duplicated()].
Pick one occurrence explicitly: df.loc[:, df.columns == "a"].iloc[:, 0].
Rename columns to unique names right after the concat/join that created duplicates.

Example fix

// before
pl.Series(pdf["a"])  # pdf has two "a" columns

// after
pdf = pdf.loc[:, ~pdf.columns.duplicated()]
pl.Series(pdf["a"])

Defensive patterns

Strategy: validation

Validate before calling

obj = pdf[col]
if getattr(obj, "ndim", 1) > 1:  # duplicated label -> DataFrame
    raise ValueError(f"duplicate pandas columns: {obj.columns.tolist()}")
s = pl.Series(obj)

Type guard

def is_unique_pandas_column(pdf: pd.DataFrame, col: str) -> bool:
    return (pdf.columns == col).sum() == 1

Try / catch

try:
    s = pl.Series(pdf[col])
except ValueError as e:
    if "duplicate column names found" in str(e):
        pdf = pdf.loc[:, ~pdf.columns.duplicated()]
        s = pl.Series(pdf[col])
    else:
        raise

Prevention

Deduplicate right after any concat/join: pdf.loc[:, ~pdf.columns.duplicated()].
Assert pdf.columns.is_unique before column access loops.
Use df.loc[:, label].iloc[:, k] when duplicates are intentional.

When it happens

Trigger: pd.DataFrame with two columns both named "a" (after concat/join/rename), then pl.Series(df["a"]) or pl.from_pandas(df["a"]).

Common situations: pd.concat(axis=1) or joins that repeat column labels; CSVs with repeated headers combined with rename operations that collapse names; generic code doing df[col] over user-supplied col names.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/041666f34117dc81. Report an issue: GitHub.