pola-rs/polars · error · NotImplementedError

`collect_all` is not supported by

Error message

`collect_all` is not supported by {type(self).__name__}

What it means

`collect_all` runs several LazyFrames in parallel; it is an optional capability on the `Engine` interface. The base class raises NotImplementedError with this message for engines that do not implement multi-query parallel collection.

Solutions

  1. Use the default engine (omit `engine`) which implements `collect_all`.
  2. Loop over frames with `lf.collect()` per frame instead of `collect_all`.
  3. Implement `collect_all` in your custom Engine to fan queries out to the backend.
  4. Confirm the chosen engine's supported methods in polars documentation before using batch APIs.

Example fix

// before
dfs = pl.collect_all(lfs, engine="my-engine")
// after
dfs = pl.collect_all(lfs)  # default engine
Defensive patterns

Strategy: try-catch

Validate before calling

if type(engine_obj).collect_all is Engine.collect_all:
    engine = None  # use default

Type guard

def supports_collect_all(engine: object) -> bool:
    return type(engine).collect_all is not Engine.collect_all

Try / catch

try:
    dfs = pl.collect_all(lfs, engine=eng)
except NotImplementedError:
    dfs = [lf.collect() for lf in lfs]

Prevention

When it happens

Trigger: Calling `pl.collect_all(lfs, engine=<engine>)` or `LazyFrame.collect_all` with an engine that lacks the `collect_all` override (e.g. a custom engine).

Common situations: Batch-reporting scripts collecting many frames at once with a custom or restricted engine; migrating code between engines where the new one lacks `collect_all`.

Related errors

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/529d7313865abd5c. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/lazyframe/engine.py:171

    def collect_batches(
        self,
        lf: LazyFrame,
        *,
        optimizations: QueryOptFlags,
        maintain_order: bool = True,
        chunk_size: int | None = None,
        lazy: bool = False,
    ) -> _CollectBatches:
        """Execute `lf`, yielding its result in batches."""
        msg = f"`collect_batches` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)

    def collect_all(
        self, lfs: Iterable[LazyFrame], *, optimizations: QueryOptFlags
    ) -> list[DataFrame]:
        """Execute several queries, potentially in parallel."""
        msg = f"`collect_all` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)

    def collect_all_async(
        self,
        lfs: Iterable[LazyFrame],
        *,
        optimizations: QueryOptFlags,
        gevent: bool = False,
    ) -> AsyncResult[list[DataFrame]]:
        """Execute several queries asynchronously."""
        msg = f"`collect_all_async` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)

    def sink_parquet(
        self,
        lf: LazyFrame,
        path: str | Path | IO[bytes] | PartitionBy,
        *,
        compression: ParquetCompression,

View on GitHub (pinned to fe841f959e)