pola-rs/polars · error · ValueError

'infer_schema_length' should be positive

Error message

'infer_schema_length' should be positive

What it means

Raised by pl.scan_ndjson (and pl.read_ndjson, which wraps it) when infer_schema_length is exactly 0. NDJSON schema inference uses sampled records to build the schema, so a zero-length sample is meaningless; unlike some CSV paths where 0 disables inference, polars requires this value to be positive (or None). The check runs after source normalization and before any scan work starts.

Source

Thrown at py-polars/src/polars/io/ndjson.py:319

        Include the path of the source file(s) as a column with this name.
    """
    sources: list[str] | list[Path] | list[IO[str]] | list[IO[bytes]] = []
    if isinstance(source, (str, Path)):
        source = normalize_filepath(source, check_not_directory=False)
    elif isinstance(source, list):
        if is_path_or_str_sequence(source):
            sources = [
                normalize_filepath(source, check_not_directory=False)
                for source in source
            ]
        else:
            sources = source

        source = None  # type: ignore[assignment]

    if infer_schema_length == 0:
        msg = "'infer_schema_length' should be positive"
        raise ValueError(msg)

    if retries is not None:
        msg = "the `retries` parameter was deprecated in 1.37.1; specify 'max_retries' in `storage_options` instead."
        issue_deprecation_warning(msg)
        storage_options = storage_options or {}
        storage_options["max_retries"] = retries

    if file_cache_ttl is not None:
        msg = "file cache is no longer supported as of 1.39.0."
        issue_deprecation_warning(msg)

    credential_provider_builder = _init_credential_provider_builder(
        credential_provider, source, storage_options, "scan_ndjson"
    )

    del credential_provider

    pylf = PyLazyFrame.new_from_ndjson(

View on GitHub (pinned to df599052da)

Solutions

1. Pass a positive integer such as infer_schema_length=100 (the default) to sample that many rows.
2. Pass infer_schema_length=None to scan every record for schema inference when you need the full-file schema.
3. If you intended 'do not infer', instead pass an explicit schema via schema_overrides/schema so inference is not needed.
4. Trace where the 0 comes from (CLI arg, config, computed value) and clamp it to a positive default before calling polars.

Example fix

// before
pl.read_ndjson('events.ndjson', infer_schema_length=0)  # ValueError

// after
pl.read_ndjson('events.ndjson', infer_schema_length=100)  # or None to infer from all rows

def safe_infer_schema_length(n):
    if n == 0:
        return None  # infer from all rows, or a positive default like 100
    return n

pl.read_ndjson(path, infer_schema_length=safe_infer_schema_length(n))

def is_valid_infer_schema_length(n: object) -> bool:
    return n is None or (isinstance(n, int) and not isinstance(n, bool) and n > 0)

try:
    df = pl.read_ndjson(path, infer_schema_length=n)
except ValueError as e:
    if 'infer_schema_length' in str(e):
        n = None
        df = pl.read_ndjson(path, infer_schema_length=n)
    else:
        raise

When it happens

Trigger: Calling pl.read_ndjson('data.ndjson', infer_schema_length=0) or pl.scan_ndjson(..., infer_schema_length=0). Also happens when infer_schema_length is computed (e.g. min(len(preview), 0) or a config value that resolves to 0) and passed through unchanged.

Common situations: Copy-pasting CSV-reading code where infer_schema_length=0 was used to mean 'take all rows' or 'no inference'; downstream code that derives the value from an empty sample or a CLI flag defaulting to 0; refactors that changed None to 0 assuming they are equivalent.

Related errors

• There is no natural representation of DayTime in JSON.
• Deserialization from JSON not implemented for {adt:?}
• expected a file path; {path!r} is a directory
• expected list or dict of objects
• invalid `return_type`; found {return_type!r}, expected one o

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/130ba817e929daa5. Report an issue: GitHub.