pola-rs/polars · error · DuplicateError

iterable passed to pl.Schema contained duplicate name '{name

Error message

iterable passed to pl.Schema contained duplicate name '{name}'

What it means

When pl.Schema is built from an iterable of (name, dtype) pairs (including Arrow field exports), a repeated column name raises polars.exceptions.DuplicateError because an ordered mapping requires unique keys. Mapping input cannot trigger it since a dict is already deduplicated.

Source

Thrown at py-polars/src/polars/schema.py:145

    ) -> None:
        if _is_arrow_schema_exportable(schema) and not isinstance(schema, Schema):
            init_polars_schema_from_arrow_c_schema(self, schema)
            return

        # `Mapping[tuple[str, SchemaInitDataType]]` is not valid at runtime, even
        # though it is a `Iterable[tuple[str, SchemaInitDataType]]`.
        input: Iterable[tuple[str, SchemaInitDataType] | ArrowSchemaExportable]
        input = schema.items() if isinstance(schema, Mapping) else (schema or ())  # type: ignore[assignment]
        for v in input:
            name, tp = (
                polars_schema_field_from_arrow_c_schema(v)
                if _is_arrow_schema_exportable(v)
                else v
            )

            if name in self:
                msg = f"iterable passed to pl.Schema contained duplicate name '{name}'"
                raise DuplicateError(msg)

            if not check_dtypes:
                super().__setitem__(name, tp)  # type: ignore[assignment]
            elif is_polars_dtype(tp):
                super().__setitem__(name, _check_dtype(tp))
            else:
                self[name] = tp

    def __eq__(self, other: object) -> bool:
        if not isinstance(other, Mapping):
            return False
        if len(self) != len(other):
            return False
        for (nm1, tp1), (nm2, tp2) in zip(self.items(), other.items(), strict=True):
            if nm1 != nm2 or not tp1.is_(tp2):
                return False
        return True

View on GitHub (pinned to df599052da)

Solutions

1. Deduplicate with last-wins semantics before constructing: pl.Schema(dict(items))
2. Find and fix the upstream duplication: [n for n in set(names) if names.count(n) > 1]
3. Rename the duplicate if both columns are legitimately distinct

Example fix

# before
pl.Schema([('a', pl.Int8), ('a', pl.Int64)])  # DuplicateError

# after
pl.Schema(dict([('a', pl.Int8), ('a', pl.Int64)]))  # last wins -> {'a': Int64}

names = [n for n, _ in items]
if len(set(names)) != len(names):
    dupes = {n for n in names if names.count(n) > 1}
    raise ValueError(f'duplicate schema names: {sorted(dupes)}')
schema = pl.Schema(items)

When it happens

Trigger: pl.Schema([('a', pl.Int8), ('a', pl.Int64)]); zipping names/dtypes lists where names repeat; concatenating schema items from multiple frames into one list.

Common situations: Merging schemas from several sources (ETL column alignment); upstream data with duplicated column names; zip of columns and inferred types drifting out of sync.

Related errors

• `Expr.str.json_decode` needs an explicitly given `dtype` oth
• `columns` arg should only have unique values, got {columns!r
• `schema_overrides` should be of type list or dict
• `schema_overrides` should be of type list or dict, got {qual
• cannot describe a LazyFrame that has no columns

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/5077e4f7ebca12eb. Report an issue: GitHub.