pola-rs/polars · error · ValueError

number of rows per chunk must be >= 1

Error message

number of rows per chunk must be >= 1

What it means

`pl.Config.set_streaming_chunk_size(size)` sets POLARS_STREAMING_CHUNK_SIZE, the number of rows each thread processes per chunk in the streaming engine. Values must be >= 1 (or None to let polars auto-tune); 0 and negatives raise ValueError.

Source

Thrown at py-polars/src/polars/config.py:960

        Overwrite chunk size used in `streaming` engine.

        By default, the chunk size is determined by the schema
        and size of the thread pool. For some datasets (esp.
        when you have large string elements) this can be too
        optimistic and lead to Out of Memory errors.

        Parameters
        ----------
        size
            Number of rows per chunk. Every thread will process chunks
            of this size.
        """
        if size is None:
            os.environ.pop("POLARS_IDEAL_MORSEL_SIZE", None)
        else:
            if size < 1:
                msg = "number of rows per chunk must be >= 1"
                raise ValueError(msg)

            os.environ["POLARS_IDEAL_MORSEL_SIZE"] = str(size)
        plr.config_reload_env_var("POLARS_IDEAL_MORSEL_SIZE")
        return cls

    @classmethod
    def set_tbl_cell_alignment(cls, format: Alignment | None) -> type[Config]:
        """
        Set table cell alignment.

        Parameters
        ----------
        format : str
            * "LEFT": left aligned
            * "CENTER": center aligned
            * "RIGHT": right aligned

        Examples

View on GitHub (pinned to 68506541d2)

Solutions

1. Pass a positive integer such as 10_000, or None to use polars' automatic chunk size.
2. Guard computed sizes: `size = max(1, computed)`.
3. Prefer not setting it at all unless profiling showed a benefit.

Example fix

# before
pl.Config.set_streaming_chunk_size(len(batch) // n_threads)  # 0 for small batches

# after
pl.Config.set_streaming_chunk_size(max(1, len(batch) // n_threads))

def set_chunk_size(size: int | None) -> None:
    if size is not None and size < 1:
        raise ValueError("streaming chunk size must be >= 1 (or None for auto)")
    pl.Config.set_streaming_chunk_size(size)

When it happens

Trigger: `pl.Config.set_streaming_chunk_size(0)`; passing a size computed from data, e.g. `len(df) // n_threads`, which evaluates to 0 for small inputs.

Common situations: Tuning streaming memory footprint in benchmarks/ETL; deriving chunk size from a row count that can legitimately be 0.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• `separator` must be a single character; found {separator!r}
• number of characters must be > 0
• invalid alignment: {format!r}
• invalid table format name: {format!r} Expected one of: {', '
• limit should be positive

AI-assisted analysis of pola-rs/polars@68506541d2 (2026-09-10). Data as JSON: /api/errors/ce07129da93d5bfe. Report an issue: GitHub.