pola-rs/polars · error · ValueError

Cannot set `iter_batches` for {self.driver_name} without als

Error message

Cannot set `iter_batches` for {self.driver_name} without also setting a non-zero `batch_size`

What it means

On the Arrow fetch path (_from_arrow), some drivers are registered with exact_batch_size=True, meaning their batched fetch method (e.g. fetch_arrow_batches / turbodbc's fetchallarrow with size) requires an explicit row count per batch. If you request iter_batches=True on such a driver without a non-zero batch_size, polars cannot construct the call and raises this ValueError naming the driver. This is raised inside the driver-properties loop after version checks, before any data is fetched.

Source

Thrown at py-polars/src/polars/io/database/_executor.py:246

                        "adbc_driver_manager" if driver == "adbc" else self.driver_name
                    )
                    # if the minimum version constraint is not met, try additional
                    # driver properties with lower constraints
                    try:
                        self._check_module_version(driver_to_check, ver)
                    except ModuleUpgradeRequiredError:
                        if i < len(driver_properties_list):
                            continue
                        raise

                if iter_batches and (
                    driver_properties["exact_batch_size"] and not batch_size
                ):
                    msg = (
                        f"Cannot set `iter_batches` for {self.driver_name} "
                        "without also setting a non-zero `batch_size`"
                    )
                    raise ValueError(msg)  # noqa: TRY301

                frames = (
                    self._apply_overrides(batch, (schema_overrides or {}))
                    if isinstance(batch, DataFrame)
                    else DataFrame(batch)
                    for batch in self._fetch_arrow(
                        driver_properties,
                        iter_batches=iter_batches,
                        batch_size=batch_size,
                    )
                )
                return frames if iter_batches else next(frames)  # type: ignore[arg-type,return-value]
        except Exception as err:
            # eg: valid turbodbc/snowflake connection, but no arrow support
            # compiled in to the underlying driver (or on this connection)
            arrow_not_supported = (
                "does not support Apache Arrow",
                "Apache Arrow format is not supported",

View on GitHub (pinned to df599052da)

Solutions

Pass a concrete batch size together with iter_batches: batch_size=50_000
Drop iter_batches to read the whole result as one Arrow table if memory allows
If you truly need unbounded streaming, switch to a driver/cursor whose fetch path does not require exact batch sizes

Example fix

# before
for df in pl.read_database(query, connection=conn, iter_batches=True):
    ...

# after
for df in pl.read_database(query, connection=conn, iter_batches=True,
                           batch_size=50_000):
    ...

Defensive patterns

Strategy: validation

Validate before calling

if iter_batches and not batch_size:
    batch_size = 50_000  # or raise, per your policy
for df in pl.read_database(query, connection=conn,
                           iter_batches=iter_batches,
                           batch_size=batch_size):
    ...

Prevention

Treat iter_batches and batch_size as one coupled option - always set them together in helpers
Default batch_size explicitly at the API boundary of your data layer instead of relying on library defaults
Check the driver registry requirements for your driver before promising streaming reads

When it happens

Trigger: pl.read_database(q, connection=turbodbc_connection, iter_batches=True) with no batch_size; same for ADBC connections whose registry entry sets exact_batch_size and batch_size omitted or 0.

Common situations: Memory-conscious streaming reads where the author assumed polars would pick a default batch size; refactoring a working call by removing batch_size while keeping iter_batches.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/711c8a519f44edf0. Report an issue: GitHub.