pola-rs/polars · error · ValueError

cannot use glob patterns and integer based projection as `co

Error message

cannot use glob patterns and integer based projection as `columns` argument

Use columns: List[str]

What it means

When read_csv receives a glob pattern as source, it internally delegates to scan_csv (functions.py:686) and applies selection via LazyFrame.select, which only accepts column names. Integer-based projection (positional column indices) is therefore unsupported on glob sources, because column 0 of each matched file could differ. Passing a list of ints as columns raises this ValueError.

Source

Thrown at py-polars/src/polars/io/csv/functions.py:721

            skip_rows_after_header=skip_rows_after_header,
            row_index_name=row_index_name,
            row_index_offset=row_index_offset,
            eol_char=eol_char,
            raise_if_empty=raise_if_empty,
            truncate_ragged_lines=truncate_ragged_lines,
            decimal_comma=decimal_comma,
            glob=glob,
        )
        if columns is None:
            return scan._collect_eager()
        elif is_str_sequence(columns, allow_str=False):
            return scan.select(columns)._collect_eager()
        else:
            msg = (
                "cannot use glob patterns and integer based projection as `columns` argument"
                "\n\nUse columns: List[str]"
            )
            raise ValueError(msg)

    projection, columns = parse_columns_arg(columns)

    pydf = PyDataFrame.read_csv(
        source,
        infer_schema_length,
        batch_size,
        has_header,
        ignore_errors,
        n_rows,
        skip_rows,
        skip_lines,
        projection,
        separator,
        rechunk,
        columns,
        encoding,
        n_threads,

View on GitHub (pinned to df599052da)

Solutions

Select by name instead: columns=['ts', 'value'] (headers must match across all matched files)
Expand the glob yourself and read positionally per file: pl.concat([pl.read_csv(f, columns=[0, 2]) for f in sorted(glob.glob('data/2024-*.csv'))])
Read without columns and slice positionally afterwards: pl.read_csv('data/2024-*.csv').select(pl.nth([0, 2]))

Example fix

# before
pl.read_csv('data/2024-*.csv', columns=[0, 2])

# after - select by name
pl.read_csv('data/2024-*.csv', columns=['ts', 'value'])

# after - expand glob for positional reads
import glob
pl.concat([pl.read_csv(f, columns=[0, 2]) for f in sorted(glob.glob('data/2024-*.csv'))])

Defensive patterns

Strategy: validation

Validate before calling

import glob as globmod
from polars.utils.various import is_glob_pattern

if is_glob_pattern(source) and columns and isinstance(columns[0], int):
    files = sorted(globmod.glob(source))
    df = pl.concat([pl.read_csv(f, columns=columns) for f in files])
else:
    df = pl.read_csv(source, columns=columns)

Type guard

def glob_safe_columns(source: str, columns) -> list[str] | None:
    """Reject int projections for glob sources before calling read_csv."""
    if columns and is_glob_pattern(source) and isinstance(columns[0], int):
        raise TypeError('use column names (or expand the glob) for glob sources')
    return columns

Prevention

Prefer column names over positional indices in any code path that may receive globs
Centralize glob handling: expand files once and read each file positionally, then concat
If shard headers are unreliable, normalize them upstream so name-based selection is safe

When it happens

Trigger: pl.read_csv('data/2024-*.csv', columns=[0, 2]); any source string where is_glob_pattern(source) is true (contains * ? [ ]) combined with an int sequence for columns; also fires when columns is a non-string sequence after the is_str_sequence check fails.

Common situations: Seasonal/partitioned exports read as one glob; porting single-file code that used positional columns=[0, 1] to a wildcard path; files whose header names differ across shards so names cannot be used.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/a30dd38c0ca43d01. Report an issue: GitHub.