pola-rs/polars · error · ValueError

'infer_schema_length' should be positive

Error message

'infer_schema_length' should be positive

What it means

scan_ndjson requires a positive infer_schema_length because 0 would mean sampling no rows to infer the NDJSON schema, which the engine forbids. Passing 0 is treated as an invalid argument rather than 'infer nothing'.

Solutions

1. Set infer_schema_length to a positive integer (e.g. 100 or the default 100)
2. Pass infer_schema_length=None to scan all rows for the schema
3. Clamp computed values: max(1, computed_length)

Example fix

// before
pl.read_ndjson('data.ndjson', infer_schema_length=0)
// after
pl.read_ndjson('data.ndjson', infer_schema_length=None)  # or a positive int

if infer_schema_length is not None and infer_schema_length < 1:
    raise ValueError('infer_schema_length must be a positive int or None')

try:
    lf = pl.scan_ndjson(path, infer_schema_length=n)
except ValueError as e:
    if 'infer_schema_length' in str(e):
        lf = pl.scan_ndjson(path, infer_schema_length=None)

When it happens

Trigger: pl.scan_ndjson(..., infer_schema_length=0) (also via pl.read_ndjson which forwards to scan_ndjson).

Common situations: Developers passing 0 intending 'skip inference' or copying a default of 0 from another library; dynamically computed schema lengths that evaluate to 0.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• at least one of ('key', 'max_rows_per_file'…
• cannot use 'include_key' without specifying 'key'
• Categorical physical must be one of pl.UInt(8|16|32)
• Enum categories must be unique; found duplicate
• Enum categories must not contain null values

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/130ba817e929daa5. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/io/ndjson.py:325

        Include the path of the source file(s) as a column with this name.
    """
    sources: list[str] | list[Path] | list[IO[str]] | list[IO[bytes]] = []
    if isinstance(source, (str, Path)):
        source = normalize_filepath(source, check_not_directory=False)
    elif isinstance(source, list):
        if is_path_or_str_sequence(source):
            sources = [
                normalize_filepath(source, check_not_directory=False)
                for source in source
            ]
        else:
            sources = source

        source = None  # type: ignore[assignment]

    if infer_schema_length == 0:
        msg = "'infer_schema_length' should be positive"
        raise ValueError(msg)

    credential_provider_builder = _init_credential_provider_builder(
        credential_provider, source, storage_options, "scan_ndjson"
    )

    del credential_provider

    pylf = PyLazyFrame.new_from_ndjson(
        source,
        sources,
        infer_schema_length=infer_schema_length,
        schema=schema,
        schema_overrides=schema_overrides,
        batch_size=batch_size,
        n_rows=n_rows,
        low_memory=low_memory,
        rechunk=False,
        row_index=parse_row_index_args(row_index_name, row_index_offset),

View on GitHub (pinned to fe841f959e)