pola-rs/polars · error · CopyNotAllowedError

unevenly chunked columns must be rechunked

Error message

unevenly chunked columns must be rechunked

What it means

Raised inside PolarsDataFrame._get_chunks_from_col_chunks when slicing the frame by column-0 chunk boundaries produces a slice whose columns have uneven chunking, and allow_copy=False. Aligning the columns of each slice requires rechunk (a copy), so the zero-copy contract triggers CopyNotAllowedError.

Source

Thrown at py-polars/src/polars/interchange/dataframe.py:227

    def _get_chunks_from_col_chunks(self) -> Iterator[DataFrame]:
        """
        Return chunks of this dataframe according to the chunks of the first column.

        If columns are not all chunked identically, they will be rechunked like the
        first column. If copy is not allowed, this raises a RuntimeError.
        """
        col_chunks = self.get_column(0).get_chunks()
        chunk_sizes = [chunk.size() for chunk in col_chunks]
        starts = [0] + list(accumulate(chunk_sizes))

        for i in range(len(starts) - 1):
            start, end = starts[i : i + 2]
            chunk = self._df[start:end, :]

            if not all(x == 1 for x in chunk.n_chunks("all")):
                if not self._allow_copy:
                    msg = "unevenly chunked columns must be rechunked"
                    raise CopyNotAllowedError(msg)
                chunk = chunk.rechunk()

            yield chunk

View on GitHub (pinned to df599052da)

Solutions

1. Rechunk the frame before interchange: df = df.rechunk()
2. Pass allow_copy=True so polars may align the chunking
3. Avoid producing uneven chunk layouts (rechunk after concat/join-heavy construction)

Example fix

// before
for chunk in df.__dataframe__(allow_copy=False).get_chunks():
    ...
// after
df = df.rechunk()
for chunk in df.__dataframe__(allow_copy=False).get_chunks():
    ...

if not all(n == 1 for n in df.n_chunks('all')):
    df = df.rechunk()  # required before allow_copy=False chunk iteration

def is_uniformly_chunked(df) -> bool:
    sizes = set(df.n_chunks('all'))
    return len(sizes) <= 1

from polars.interchange.protocol import CopyNotAllowedError
try:
    for chunk in df.__dataframe__(allow_copy=False).get_chunks():
        ...
except CopyNotAllowedError:
    for chunk in df.rechunk().__dataframe__(allow_copy=False).get_chunks():
        ...

When it happens

Trigger: Calling __dataframe__(allow_copy=False) followed by get_chunks() on a frame whose columns were chunked independently (e.g. after joins, with_columns on multi-chunk frames, hstack of differently-chunked columns); per-column concat or updates leaving misaligned chunk layouts.

Common situations: Incrementally built frames (concat/join/update) handed to strict zero-copy interchange consumers; ETL stages that append columns without rechunking; interchange adapters that require allow_copy=False for memory guarantees.

Related errors

• non-contiguous buffer must be made contiguous
• string buffers must be converted
• data buffer must be cast from {data_dtype} to UInt32
• byte-packed boolean buffer must be converted to bit-packed b
• offsets buffer must be cast from {polars_dtype} to Int64

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/ea8e0ba3b778dd98. Report an issue: GitHub.