pola-rs/polars · error

cannot use `partition_by` with `maintain_order=False, includ

Error message

cannot use `partition_by` with `maintain_order=False, include_key=False, as_dict=True`

What it means

DataFrame.partition_by(as_dict=True) builds a dict mapping group keys to partition frames by pairing each partition with a key. When include_key=False the key columns are stripped from the partitions, so the keys must be recovered separately from self.select(by).unique(maintain_order=True) and zipped positionally — which is only sound if partitions come back in a deterministic order. With maintain_order=False the Rust-side partition order is not guaranteed to match, so polars rejects the exact combination (maintain_order=False, include_key=False, as_dict=True) up front.

Source

Thrown at py-polars/src/polars/dataframe/frame.py:10214

        │ str ┆ i64 ┆ i64 │
        ╞═════╪═════╪═════╡
        │ c   ┆ 3   ┆ 1   │
        └─────┴─────┴─────┘}
        """
        by_parsed = _expand_selectors(self, by, *more_by)

        partitions = [
            self._from_pydf(_df)
            for _df in self._df.partition_by(by_parsed, maintain_order, include_key)
        ]

        if as_dict:
            if include_key:
                names = [p.select(by_parsed).row(0) for p in partitions]
            else:
                if not maintain_order:  # Group keys cannot be matched to partitions
                    msg = "cannot use `partition_by` with `maintain_order=False, include_key=False, as_dict=True`"
                    raise ValueError(msg)
                names = self.select(by_parsed).unique(maintain_order=True).rows()

            return dict(zip(names, partitions, strict=True))

        return partitions

    def shift(self, n: int = 1, *, fill_value: IntoExpr | None = None) -> DataFrame:
        """
        Shift values by the given number of indices.

        Parameters
        ----------
        n
            Number of indices to shift forward. If a negative value is passed, values
            are shifted in the opposite direction instead.
        fill_value
            Fill the resulting null values with this value. Accepts scalar expression
            input. Non-expression inputs are parsed as literals.

View on GitHub (pinned to df599052da)

Solutions

1. Re-enable order: partition_by('a', maintain_order=True, include_key=False, as_dict=True)
2. Keep key columns and strip them afterwards: partition_by('a', maintain_order=False, include_key=True, as_dict=True), then pop/drop the key cols per partition
3. Return a list and derive keys yourself with deterministic ordering on both sides (select(by).unique(maintain_order=True) plus partition_by(..., maintain_order=True))

Example fix

# before
parts = df.partition_by('a', maintain_order=False, include_key=False, as_dict=True)

# after
parts = df.partition_by('a', maintain_order=True, include_key=False, as_dict=True)
# or keep keys and strip them per partition:
# parts = {k: p.drop('a'): ...}

if as_dict and not include_key and not maintain_order:
    raise ValueError('partition_by: set maintain_order=True or include_key=True when as_dict=True')
parts = df.partition_by(by, maintain_order=maintain_order, include_key=include_key, as_dict=as_dict)

When it happens

Trigger: df.partition_by('a', maintain_order=False, include_key=False, as_dict=True) — all three flags together; performance tuning that disabled maintain_order on an existing as_dict=True, include_key=False call site.

Common situations: Building {key: sub-frame} lookup tables for per-group export/sharding while optimizing partition_by speed; dropping key columns to save memory and then relying on dict keys for identification.

Related errors

• Can not set `ascii_tables` and `tbl_formatting` at the same
• invalid `return_type`; found {return_type!r}, expected one o
• `offset` input for `with_row_index` cannot be {issue}, got {
• unexpected input for `strategy`: {strategy!r} Choose one of
• cannot specify both `n` and `fraction`

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/0df560876dcfe441. Report an issue: GitHub.