pola-rs/polars · error · TypeError

`schema_overrides` should be of type list or dict

Error message

`schema_overrides` should be of type list or dict

What it means

read_csv accepts schema_overrides only as a dict {name: dtype} or a list of dtypes; the isinstance(schema_overrides, (dict, Sequence)) check (py-polars/src/polars/io/csv/functions.py:320-324) raises TypeError for anything else - a bare dtype, a set, a generator, or a numpy array are not Sequences per collections.abc.

Source

Thrown at py-polars/src/polars/io/csv/functions.py:323

    _check_arg_is_1byte("eol_char", eol_char, can_be_empty=False)

    projection, columns = parse_columns_arg(columns)
    storage_options = storage_options or {}

    if columns and not has_header:
        for column in columns:
            if not column.startswith("column_"):
                msg = (
                    "specified column names do not start with 'column_',"
                    " but autogenerated header names were requested"
                )
                raise ValueError(msg)

    if schema_overrides is not None and not isinstance(
        schema_overrides, (dict, Sequence)
    ):
        msg = "`schema_overrides` should be of type list or dict"
        raise TypeError(msg)

    if (
        use_pyarrow
        and schema_overrides is None
        and n_rows is None
        and n_threads is None
        and not low_memory
        and null_values is None
    ):
        include_columns: Sequence[str] | None = None
        if columns:
            if not has_header:
                # Convert 'column_1', 'column_2', ... column names to 'f0', 'f1', ...
                # column names for pyarrow, if CSV file does not contain a header.
                include_columns = [f"f{int(column[7:]) - 1}" for column in columns]
            else:
                include_columns = columns

View on GitHub (pinned to df599052da)

Solutions

  1. Use a dict for named columns: schema_overrides={'a': pl.Int64}
  2. Use a list for positional dtypes: schema_overrides=[pl.Int64, pl.Utf8]
  3. Consume generators/sets into a list before the call: list(schema_overrides)
  4. For a whole-file single dtype, use the `dtypes`-style parameter or cast after reading

Example fix

# before
df = pl.read_csv("f.csv", schema_overrides=pl.Int64)
# after
df = pl.read_csv("f.csv", schema_overrides={"a": pl.Int64, "b": pl.Int64})
Defensive patterns

Strategy: type-guard

Validate before calling

from collections.abc import Sequence

def normalize_overrides(schema_overrides):
    if isinstance(schema_overrides, dict) or (
        isinstance(schema_overrides, (list, tuple))
        and all(isinstance(dt, pl.DataType) or True for dt in schema_overrides)
    ):
        return schema_overrides
    if isinstance(schema_overrides, (set, frozenset)) or hasattr(schema_overrides, "__next__"):
        return list(schema_overrides)
    raise TypeError("schema_overrides must be a dict or a list of dtypes")

Type guard

from collections.abc import Sequence

def is_valid_schema_overrides(o: object) -> bool:
    return isinstance(o, (dict, list, tuple)) and not isinstance(o, (str, bytes))

Try / catch

try:
    df = pl.read_csv(path, schema_overrides=ov)
except TypeError as e:
    if "schema_overrides" in str(e):
        df = pl.read_csv(path, schema_overrides=list(ov))
    else:
        raise

Prevention

When it happens

Trigger: pl.read_csv(f, schema_overrides=pl.Int64) (a single dtype instead of a per-column mapping); schema_overrides={'a','b'} (set); a generator/iterator of dtypes; a numpy object array of dtypes.

Common situations: Trying to apply one dtype to the whole file (not supported - overrides are per column); configs delivering dtypes as comma-separated strings or iterables; building overrides with a set comprehension.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/35710aa11fb6ea8f. Report an issue: GitHub.