pola-rs/polars · error · ValueError

`n_chunks` must be a multiple of the number of chunks of…

Error message

`n_chunks` must be a multiple of the number of chunks of this dataframe ({total_n_chunks})

What it means

Raised by PolarsDataFrame.get_chunks(n_chunks=...) in the dataframe interchange implementation. The method can only split a frame by dividing every existing chunk into an equal number of sub-chunks, so n_chunks must be a positive multiple of the frame's current chunk count (df.num_chunks()); values <= 0, or values that are not exact multiples, raise this ValueError. Passing None or exactly num_chunks() always works because it takes the no-split path.

Solutions

1. Pass n_chunks=None (or exactly icdf.num_chunks()) to iterate the existing chunks unchanged.
2. Pass a positive multiple of the chunk count, e.g. n_chunks=icdf.num_chunks() * k.
3. Rechunk the source before interchange: df = df.rechunk() (or pl.concat(..., rechunk=True)) so num_chunks() == 1 and any positive n_chunks is valid.

Example fix

# before
icdf = df.__dataframe__()
for chunk in icdf.get_chunks(n_chunks=4):  # ValueError when df has 3 chunks
    ...

# after
icdf = df.rechunk().__dataframe__()  # single chunk -> any positive n_chunks works
for chunk in icdf.get_chunks(n_chunks=4):
    ...

total = icdf.num_chunks()
if n_chunks is None or n_chunks <= 0 or n_chunks % total != 0:
    n_chunks = total  # fall back to the frame's natural chunk count
for chunk in icdf.get_chunks(n_chunks=n_chunks):
    ...

try:
    chunks = list(icdf.get_chunks(n_chunks=k))
except ValueError as e:
    if "`n_chunks` must be a multiple" not in str(e):
        raise
    chunks = list(icdf.get_chunks(n_chunks=icdf.num_chunks()))  # retry with a valid value

When it happens

Trigger: Calling icdf.get_chunks(n_chunks=k) where the backing pl.DataFrame has total_n_chunks = icdf.num_chunks() chunks and k <= 0 or k % total_n_chunks != 0 (e.g. a 3-chunk frame with n_chunks=2, or n_chunks=0). Interchange consumers such as other libraries calling df.__dataframe__().get_chunks(n_chunks=batch_size) hit this when the source frame is chunked.

Common situations: Hardcoding a batch/partition count in generic interop code; receiving a multi-chunk frame from pl.concat, scans, or chunked ingestion and assuming any n_chunks is accepted (any positive value works only when num_chunks() == 1); passing 0 hoping to mean 'default'.

Related errors

• data type not supported by the interchange protocol
• bitmask must be constructed
• bitmask must be inverted
• byte-packed boolean buffer must be converted to bit-packed…
• bytemask must be converted into a bitmask

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/04a0a4b45cfb6ff5. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/interchange/dataframe.py:194

        -----
        When the columns in the dataframe are chunked unevenly, or when `n_chunks` is
        higher than the number of chunks in the dataframe, a slice must be performed
        that is not on the chunk boundary. This will trigger some compute for columns
        that contain null values and boolean columns.
        """
        total_n_chunks = self.num_chunks()
        chunks = self._get_chunks_from_col_chunks()

        if (n_chunks is None) or (n_chunks == total_n_chunks):
            for chunk in chunks:
                yield PolarsDataFrame(chunk, allow_copy=self._allow_copy)

        elif (n_chunks <= 0) or (n_chunks % total_n_chunks != 0):
            msg = (
                "`n_chunks` must be a multiple of the number of chunks of this"
                f" dataframe ({total_n_chunks})"
            )
            raise ValueError(msg)

        else:
            subchunks_per_chunk = n_chunks // total_n_chunks
            for chunk in chunks:
                size = len(chunk)
                step = size // subchunks_per_chunk
                if size % subchunks_per_chunk != 0:
                    step += 1
                for start in range(0, step * subchunks_per_chunk, step):
                    yield PolarsDataFrame(
                        chunk[start : start + step, :],
                        allow_copy=self._allow_copy,
                    )

    def _get_chunks_from_col_chunks(self) -> Iterator[DataFrame]:
        """
        Return chunks of this dataframe according to the chunks of the first column.

View on GitHub (pinned to 9b5d73fd00)