{"record":{"id":"fdbf0532a36144d9","repo":"pola-rs/polars","slug":"pandas-indices-and-column-names-must-not-overlap","errorCode":null,"errorMessage":"Pandas indices and column names must not overlap.","messagePattern":"Pandas indices and column names must not overlap\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/_utils/construction/dataframe.py","lineNumber":1082,"sourceCode":"    stringified_index: set[str] = (\n        {str(idx) for idx in data.index.names} if include_index else set()\n    )\n\n    non_unique_cols: bool = len(stringified_cols) < len(data.columns)\n    non_unique_indices: bool = (\n        (len(stringified_index) < len(data.index.names)) if include_index else False\n    )\n    if non_unique_cols or non_unique_indices:\n        msg = (\n            \"Pandas dataframe contains non-unique indices and/or column names. \"\n            \"Polars dataframes require unique string names for columns.\"\n        )\n        raise ValueError(msg)\n\n    overlapping_cols_and_indices: set[str] = stringified_cols & stringified_index\n    if len(overlapping_cols_and_indices) > 0:\n        msg = \"Pandas indices and column names must not overlap.\"\n        raise ValueError(msg)\n\n\ndef pandas_to_pydf(\n    data: pd.DataFrame,\n    schema: SchemaDefinition | None = None,\n    *,\n    schema_overrides: SchemaDict | None = None,\n    strict: bool = True,\n    rechunk: bool = True,\n    nan_to_null: bool = True,\n    include_index: bool = False,\n) -> PyDataFrame:\n    \"\"\"Construct a PyDataFrame from a pandas DataFrame.\"\"\"\n    _check_pandas_columns(data, include_index=include_index)\n\n    convert_index = include_index and not _pandas_has_default_index(data)\n\n    if not convert_index:","sourceCodeStart":1064,"sourceCodeEnd":1100,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/_utils/construction/dataframe.py#L1064-L1100","documentation":"When converting a pandas DataFrame with index inclusion (pl.from_pandas(..., include_index=True) or equivalent constructor paths), polars must place both the index levels and the columns into one flat, uniquely-named schema. If any stringified index-level name equals a column name, the sets overlap and this ValueError is raised before conversion starts.","triggerScenarios":"`pl.from_pandas(df.set_index(\"a\"), include_index=True)` while a column 'a' also still exists (set_index without dropping duplicates); any index level whose name (including default names like 'index' or None->stringified) collides with a data column name.","commonSituations":"set_index on a column that also remains in the frame (dup='keep' style patterns); reset_index followed by partial set_index roundtrips; Polars->pandas->Polars roundtrips where the pandas index inherited the column's name; default index name 'index' colliding with a real 'index' column.","solutions":["Rename the axis before converting: `df = df.rename_axis(\"a_idx\")`","Or remove the colliding column: `df = df.drop(columns=[\"a\"])` if the index already carries that data","Use `df.reset_index(drop=True)` when the index holds nothing you need, then convert without include_index","Convert without include_index (the default) so index names are never considered"],"exampleFix":"# before\nimport pandas as pd, polars as pl\npdf = pd.DataFrame({\"a\": [1, 2], \"b\": [3, 4]}).set_index(\"a\", drop=False)\npl.from_pandas(pdf, include_index=True)  # ValueError: Pandas indices and column names must not overlap.\n\n# after\npl.from_pandas(pdf.rename_axis(\"a_idx\"), include_index=True)\n# or drop the duplicate column:\npl.from_pandas(pdf.drop(columns=[\"a\"]), include_index=True)","handlingStrategy":"validation","validationCode":"import pandas as pd\n\ndef index_cols_disjoint(df: pd.DataFrame) -> bool:\n    idx = {str(n) for n in df.index.names}\n    cols = {str(c) for c in df.columns}\n    return not (idx & cols)\n\nassert index_cols_disjoint(df), \"index level name collides with a column name\"\n# then safe: pl.from_pandas(df, include_index=True)","typeGuard":"import pandas as pd\n\ndef index_cols_disjoint(df: pd.DataFrame) -> bool:\n    \"\"\"True when no stringified index-level name equals a column name.\"\"\"\n    return not ({str(n) for n in df.index.names} & {str(c) for c in df.columns})","tryCatchPattern":"try:\n    pldf = pl.from_pandas(df, include_index=True)\nexcept ValueError as e:\n    if \"overlap\" not in str(e):\n        raise\n    pldf = pl.from_pandas(df.rename_axis(\"_idx\"), include_index=True)","preventionTips":["rename_axis (or df.index.name = ...) before include_index conversions","Be explicit about set_index(drop=...) semantics so a column never coexists with a same-named index","Check {str(n) for n in df.index.names}.isdisjoint(map(str, df.columns)) in shared conversion helpers"],"tags":["pandas-interop","include-index","name-collision","validation"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}