pola-rs/polars · error · ValueError

at least one of ('key', 'max_rows_per_file', 'approximate_by

Error message

at least one of ('key', 'max_rows_per_file', 'approximate_bytes_per_file') must be specified for PartitionBy

What it means

ValueError raised by the PartitionBy constructor (py-polars/src/polars/io/partition.py) used with sink_parquet/sink_ipc to write multiple output files. PartitionBy needs at least one slicing criterion — a key to partition on, a row cap per file, or a byte target per file. With key=None, max_rows_per_file=None, and approximate_bytes_per_file left at its 'auto' default, there is no way to decide when to start a new file, so construction fails immediately (this API is also marked unstable).

Source

Thrown at py-polars/src/polars/io/partition.py:111

        key: str | Expr | Sequence[str | Expr] | Mapping[str, Expr] | None = None,
        include_key: bool | None = None,
        max_rows_per_file: int | None = None,
        approximate_bytes_per_file: int | Literal["auto"] | None = "auto",
    ) -> None:
        msg = "`PartitionBy` functionality is considered unstable"
        issue_unstable_warning(msg)

        if (
            key is None
            and max_rows_per_file is None
            and approximate_bytes_per_file == "auto"
        ):
            msg = (
                "at least one of "
                "('key', 'max_rows_per_file', 'approximate_bytes_per_file') "
                "must be specified for PartitionBy"
            )
            raise ValueError(msg)

        if key is None and include_key is not None:
            msg = "cannot use 'include_key' without specifying 'key'"
            raise ValueError(msg)

        base_path = str(base_path)

        if approximate_bytes_per_file == "auto":
            approximate_bytes_per_file = (
                4_294_967_295 if max_rows_per_file is None else None
            )

        if approximate_bytes_per_file is None:
            approximate_bytes_per_file = (1 << 64) - 1

        self._pl_partition_by = _PartitionByInner(
            base_path=base_path,
            file_path_provider=file_path_provider,

View on GitHub (pinned to df599052da)

Solutions

1. Partition by column values: pl.PartitionBy('out/', key='year').
2. Or cap file size by rows: pl.PartitionBy('out/', max_rows_per_file=1_000_000).
3. Or cap by estimated bytes: pl.PartitionBy('out/', approximate_bytes_per_file=128_000_000).
4. If the arguments come from config, assert they are not all empty before building the PartitionBy.

Example fix

# before
pl.LazyFrame({'year': [2026, 2027]}).sink_parquet(pl.PartitionBy('data/'))

# after
pl.LazyFrame({'year': [2026, 2027]}).sink_parquet(pl.PartitionBy('data/', key='year'))

def make_partition_by(base_path, *, key=None, max_rows_per_file=None, approximate_bytes_per_file='auto', **kw):
    if key is None and max_rows_per_file is None and approximate_bytes_per_file == 'auto':
        raise ValueError('PartitionBy needs key, max_rows_per_file, or approximate_bytes_per_file')
    return pl.PartitionBy(base_path, key=key, max_rows_per_file=max_rows_per_file, approximate_bytes_per_file=approximate_bytes_per_file, **kw)

try:
    lf.sink_parquet(pl.PartitionBy(out_dir, **partition_cfg))
except ValueError as e:
    if 'must be specified for PartitionBy' in str(e):
        lf.sink_parquet(pl.PartitionBy(out_dir, **{**partition_cfg, 'max_rows_per_file': 1_000_000}))
    else:
        raise

When it happens

Trigger: pl.LazyFrame(...).sink_parquet(pl.PartitionBy('out/')) with no key, no max_rows_per_file, and no approximate_bytes_per_file — i.e. only a base_path.

Common situations: Starting from the doc example and deleting the arguments while prototyping; assuming the constructor defaults to partitioning by row count; passing variables for key/max_rows that unexpectedly evaluate to None (e.g. a config key that was never set).

Related errors

• cannot use 'include_key' without specifying 'key'
• write_parquet with `use_pyarrow=True` allows only boolean va
• write_parquet with `use_pyarrow=True` cannot be combined wit
• `sink_parquet` is not supported by {type(self).__name__}
• write_table: table format of {catalog_name}.{namespace}.{tab

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/ced195adbdb84e77. Report an issue: GitHub.