pola-rs/polars · error · DuplicateError

column {nm!r} appears more than once in the query/result cur

Error message

column {nm!r} appears more than once in the query/result cursor

What it means

While injecting inferred dtypes, ConnectionExecutor._inject_type_overrides walks the cursor description (name, type_code) pairs; if the same column name appears twice in the result set, it raises DuplicateError - a polars exception - because a DataFrame cannot hold two columns with one name. The duplicate always originates in the SQL: SELECT a, a or joins returning identically-named columns without aliases.

Source

Thrown at py-polars/src/polars/io/database/_executor.py:356

        self,
        description: list[tuple[str, Any]],
        schema_overrides: SchemaDict,
    ) -> SchemaDict:
        """
        Attempt basic dtype inference from a cursor description.

        Notes
        -----
        This is limited; the `type_code` description attr may contain almost anything,
        from strings or python types to driver-specific codes, classes, enums, etc.
        We currently only do the additional inference from string/python type values.
        (Further refinement will require per-driver module knowledge and lookups).
        """
        dupe_check = set()
        for nm, desc in description:
            if nm in dupe_check:
                msg = f"column {nm!r} appears more than once in the query/result cursor"
                raise DuplicateError(msg)
            elif desc is not None and nm not in schema_overrides:
                dtype = dtype_from_cursor_description(desc)
                if dtype is not None:
                    schema_overrides[nm] = dtype  # type: ignore[index]
            dupe_check.add(nm)

        return schema_overrides

    @staticmethod
    def _is_alchemy_async(conn: Any) -> bool:
        """Check if the given connection is SQLALchemy async."""
        try:
            from sqlalchemy.ext.asyncio import (
                AsyncConnection,
                AsyncSession,
                async_sessionmaker,
            )

View on GitHub (pinned to df599052da)

Solutions

  1. Alias every duplicated column in the SELECT list: SELECT o.id AS order_id, c.id AS customer_id
  2. Replace SELECT * with an explicit column list
  3. For dynamic SQL, programmatically de-duplicate names by appending _1, _2 suffixes when building the query

Example fix

-- before
SELECT o.id, c.id, o.total FROM orders o JOIN customers c ON o.cust_id = c.id

-- after
SELECT o.id AS order_id, c.id AS customer_id, o.total
FROM orders o JOIN customers c ON o.cust_id = c.id
Defensive patterns

Strategy: validation

Validate before calling

def dedupe_sql_columns(sql: str, describe_fn) -> str:
    names = [d[0] for d in describe_fn(sql)]  # e.g. cursor.description probe
    seen: set[str] = set()
    return sql  # build SELECT with AS aliases when len(names) != len(set(names))

# simpler: always alias join columns explicitly
QUERY = 'SELECT o.id AS order_id, c.id AS customer_id FROM orders o JOIN customers c ON o.cust_id = c.id'

Try / catch

from polars.exceptions import DuplicateError

try:
    df = pl.read_database(query, connection=conn)
except DuplicateError as err:
    dupes = {m for m in re.findall(r"'([^']+)'", str(err))}
    raise ValueError(f'alias these duplicated columns in SQL: {dupes}') from err

Prevention

When it happens

Trigger: pl.read_database('SELECT id, name, id FROM t', connection=conn); SELECT o.id, c.id FROM orders o JOIN customers c ... without aliasing; SELECT * from two tables sharing column names; SurrealDB/other drivers echoing a field twice.

Common situations: Ad-hoc joins built by string concatenation; SELECT * on wide join views; generated SQL where aliasing was forgotten; analytics views exposing duplicated metadata columns.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/50e529ceb0b56c5d. Report an issue: GitHub.