pola-rs/polars · error · NotImplementedError

`collect_all_async` is not supported by {type(self).__name__

Error message

`collect_all_async` is not supported by {type(self).__name__}

What it means

`Engine.collect_all_async` (engine.py:222) is the optional asynchronous variant of `collect_all` used by `pl.collect_all_async`; its default stub raises `NotImplementedError` with the engine class name. As with the other optional hooks, only the local engine family implements it, so `RemoteEngine` and minimal custom `Engine` subclasses reject it.

Source

Thrown at py-polars/src/polars/lazyframe/engine.py:222

        raise NotImplementedError(msg)

    def collect_all(
        self, lfs: Iterable[LazyFrame], *, optimizations: QueryOptFlags
    ) -> list[DataFrame]:
        """Execute several queries, potentially in parallel."""
        msg = f"`collect_all` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)

    def collect_all_async(
        self,
        lfs: Iterable[LazyFrame],
        *,
        optimizations: QueryOptFlags,
        gevent: bool = False,
    ) -> AsyncResult[list[DataFrame]]:
        """Execute several queries asynchronously."""
        msg = f"`collect_all_async` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)

    def sink_parquet(
        self,
        lf: LazyFrame,
        path: str | Path | IO[bytes] | PartitionBy,
        *,
        compression: ParquetCompression,
        compression_level: int | None,
        statistics: bool | str | dict[str, bool],
        row_group_size: int | None,
        data_page_size: int | None,
        maintain_order: bool,
        storage_options: StorageOptionsDict | None,
        credential_provider: CredentialProviderFunction | Literal["auto"] | None,
        retries: int | None,
        sync_on_close: SyncOnCloseMethod | None,
        metadata: ParquetMetadata | None,
        arrow_schema: ArrowSchemaExportable | None,

View on GitHub (pinned to df599052da)

Solutions

1. Use a local engine: `pl.collect_all_async(lfs, engine='streaming')`
2. Fall back to per-query background handles: `handles = [lf.collect(engine=remote, background=True) for lf in lfs]` then `fetch()` each
3. Implement `collect_all_async` on your custom `Engine` subclass
4. Feature-detect support before calling (see defense) and choose the sync path otherwise

Example fix

# before
res = pl.collect_all_async([lf1], engine=pl.RemoteEngine())  # NotImplementedError

# after
handles = [lf1.collect(engine=remote, background=True)]
dfs = [h.fetch() for h in handles]

from polars.lazyframe.engine import Engine

def supports_collect_all_async(engine: pl.Engine) -> bool:
    return type(engine).collect_all_async is not Engine.collect_all_async

if not supports_collect_all_async(engine):
    handles = [lf.collect(engine=engine, background=True) for lf in lfs]
    dfs = [h.fetch() for h in handles]

try:
    ar = pl.collect_all_async(lfs, engine=engine)
except NotImplementedError:
    handles = [lf.collect(engine=engine, background=True) for lf in lfs]

When it happens

Trigger: `pl.collect_all_async([lf1, lf2], engine=pl.RemoteEngine())` or the same with a custom engine lacking the override. Also reached when a gevent/async orchestration layer routes every query through `collect_all_async` while a non-local engine affinity is active.

Common situations: Server code that awaits many queries concurrently and is pointed at Polars Cloud by configuration; test harnesses with a fake engine implementing only the abstract methods; version upgrades where code that previously used only local engines now accepts arbitrary engine objects.

Related errors

• `collect_async` is not supported by {type(self).__name__}
• `collect_all` is not supported by {type(self).__name__}
• `collect_batches` is not supported by {type(self).__name__}
• `sink_parquet` is not supported by {type(self).__name__}
• `sink_ipc` is not supported by {type(self).__name__}

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/3250a3395cc823fa. Report an issue: GitHub.