{"record":{"id":"89cec8a2d8844e24","repo":"pola-rs/polars","slug":"more-schema-overrides-are-specified-than-there-are","errorCode":null,"errorMessage":"more schema overrides are specified than there are selected columns","messagePattern":"more schema overrides are specified than there are selected columns","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/io/csv/functions.py","lineNumber":409,"sourceCode":"            # to 'column_1', 'column_2', ...\n            tbl = tbl.rename_columns(\n                [f\"column_{int(column[1:]) + 1}\" for column in tbl.column_names]\n            )\n        elif not columns and projection:\n            # User selected columns by positional index (e.g. `columns=[0, 2]`).\n            # pyarrow's include_columns only accepts names, so the read above\n            # fetched every column; pick out the requested positions now.\n            tbl = tbl.select(list(projection))\n\n        df = pl.DataFrame._from_arrow(tbl, rechunk=rechunk)\n        if new_columns:\n            return _update_columns(df, new_columns)\n        return df\n\n    if projection and schema_overrides and isinstance(schema_overrides, list):\n        if len(projection) < len(schema_overrides):\n            msg = \"more schema overrides are specified than there are selected columns\"\n            raise ValueError(msg)\n\n        # Fix list of dtypes when used together with projection as polars CSV reader\n        # wants a list of dtypes for the x first columns before it does the projection.\n        dtypes_list: list[PolarsDataType] = [String] * (max(projection) + 1)\n\n        for idx, column_idx in enumerate(projection):\n            if idx < len(schema_overrides):\n                dtypes_list[column_idx] = schema_overrides[idx]\n\n        schema_overrides = dtypes_list\n\n    if columns and schema_overrides and isinstance(schema_overrides, list):\n        if len(columns) < len(schema_overrides):\n            msg = \"more dtypes overrides are specified than there are selected columns\"\n            raise ValueError(msg)\n\n        # Map list of dtypes when used together with selected columns as a dtypes dict\n        # so the dtypes are applied to the correct column instead of the first x","sourceCodeStart":391,"sourceCodeEnd":427,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/io/csv/functions.py#L391-L427","documentation":"When read_csv receives an integer projection (columns=[0,2]) plus a list of schema_overrides, it must expand the override list to cover positions before the projection (py-polars/src/polars/io/csv/functions.py:408-419). If len(projection) < len(schema_overrides), there are more dtypes than selected columns and no valid mapping exists, so ValueError is raised before parsing.","triggerScenarios":"pl.read_csv(f, columns=[0], schema_overrides=[pl.Int64, pl.Int64]) - one projected column but two dtypes. Dict overrides don't hit this check (they're keyed by name).","commonSituations":"Reusing a full-file dtype list after adding a columns projection as an optimization; config-driven dtype lists drifting out of sync with column selections during schema evolution.","solutions":["Trim the dtype list so there is exactly one dtype per projected column, in projection order","Switch to a dict keyed by column names so the mapping is unambiguous","Drop columns= and .select() the needed columns after reading"],"exampleFix":"# before\ndf = pl.read_csv(\"f.csv\", columns=[0], schema_overrides=[pl.Int64, pl.Int64])\n# after\ndf = pl.read_csv(\"f.csv\", columns=[0], schema_overrides=[pl.Int64])","handlingStrategy":"validation","validationCode":"def check_projection_overrides(columns, schema_overrides) -> None:\n    if (\n        isinstance(columns, list)\n        and columns\n        and all(isinstance(c, int) for c in columns)\n        and isinstance(schema_overrides, list)\n        and len(columns) < len(schema_overrides)\n    ):\n        raise ValueError(\n            f\"{len(schema_overrides)} dtypes for {len(columns)} projected columns; trim the list\"\n        )","typeGuard":null,"tryCatchPattern":"try:\n    df = pl.read_csv(path, columns=cols, schema_overrides=ov)\nexcept ValueError as e:\n    if \"more schema overrides\" in str(e) and isinstance(ov, list):\n        df = pl.read_csv(path, columns=cols, schema_overrides=ov[: len(cols)])\n    else:\n        raise","preventionTips":["Derive the dtype list and the projection from one config object","Prefer dict overrides keyed by name when selections change often","Add a length assertion in the job's parameter validation"],"tags":["polars","csv","projection","dtypes","valueerror"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}