{"record":{"id":"ddd5498b31fb8d6c","repo":"pola-rs/polars","slug":"collect-async-is-not-supported-by-type-self","errorCode":null,"errorMessage":"`collect_async` is not supported by {type(self).__name__}","messagePattern":"`collect_async` is not supported by (.+?)","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/lazyframe/engine.py","lineNumber":191,"sourceCode":"\n        This method of materializing a `LazyFrame` makes no guarantees as to where\n        the result is materialized. This can be on the GPU for the GPU-engine,\n        on the cluster or remote storage for the distributed engine and the streaming\n        engine could spill the result if it needed to.\n\n        The `QueryResult` can always be consumed as a new `LazyFrame` by calling `.lazy`\n        \"\"\"\n\n    def collect_async(\n        self,\n        lf: LazyFrame,\n        *,\n        optimizations: QueryOptFlags,\n        gevent: bool = False,\n    ) -> AsyncResult[DataFrame]:\n        \"\"\"Execute `lf` asynchronously.\"\"\"\n        msg = f\"`collect_async` is not supported by {type(self).__name__}\"\n        raise NotImplementedError(msg)\n\n    def collect_batches(\n        self,\n        lf: LazyFrame,\n        *,\n        optimizations: QueryOptFlags,\n        maintain_order: bool = True,\n        chunk_size: int | None = None,\n        lazy: bool = False,\n    ) -> Iterator[DataFrame]:\n        \"\"\"Execute `lf`, yielding its result in batches.\"\"\"\n        msg = f\"`collect_batches` is not supported by {type(self).__name__}\"\n        raise NotImplementedError(msg)\n\n    def collect_all(\n        self, lfs: Iterable[LazyFrame], *, optimizations: QueryOptFlags\n    ) -> list[DataFrame]:\n        \"\"\"Execute several queries, potentially in parallel.\"\"\"","sourceCodeStart":173,"sourceCodeEnd":209,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/lazyframe/engine.py#L173-L209","documentation":"`Engine` is the abstract base class for pluggable Polars execution backends; only `collect` and `execute` are abstract, while optional operations like `collect_async` ship a default stub that raises `NotImplementedError` naming the engine class. The built-in local engines (`in-memory`, `streaming`, `gpu`, `auto`) override it, but `RemoteEngine` and any minimal custom subclass do not. The error therefore identifies a capability gap of the specific engine instance you passed.","triggerScenarios":"`lf.collect_async(engine=pl.RemoteEngine())`, or `lf.collect_async(engine=my_engine)` where `my_engine` subclasses `pl.Engine` and only implements `collect`/`execute`. The error is raised synchronously at call time, before any work is scheduled.","commonSituations":"Migrating a pipeline to Polars Cloud and reusing gevent/async orchestration code unchanged; writing a custom engine (test double, alternative backend) and forgetting that optional `Engine` methods are not abstract; assuming every engine supports every `LazyFrame` entry point.","solutions":["Use a local engine that supports it: `lf.collect_async(engine='streaming')` (the default) or `engine='in-memory'`","For remote work use the supported background API: `handle = lf.collect(engine=remote, background=True)` or `lf.execute(...)` plus a sink","If you own the engine, implement `collect_async` on your `Engine` subclass","Catch `NotImplementedError` and degrade gracefully when the engine is chosen at runtime"],"exampleFix":"# before\nres = lf.collect_async(engine=pl.RemoteEngine())  # NotImplementedError\n\n# after\nhandle = lf.collect(engine=remote, background=True)  # InProcessQuery\ndf = handle.fetch()","handlingStrategy":"validation","validationCode":"from polars.lazyframe.engine import Engine\n\ndef engine_supports(engine: pl.Engine, method: str) -> bool:\n    # True only when the engine overrides the Engine stub\n    return getattr(type(engine), method, None) is not getattr(Engine, method)\n\nremote = pl.RemoteEngine()\nassert engine_supports(remote, 'collect_async') is False\nassert engine_supports(pl.StreamingEngine(), 'collect_async') is True","typeGuard":null,"tryCatchPattern":"try:\n    ar = lf.collect_async(engine=engine)\nexcept NotImplementedError as e:\n    # message names the engine class, e.g. 'RemoteEngine'\n    handle = lf.collect(engine=engine, background=True)","preventionTips":["Pin the engine type in code that uses async collect; make RemoteEngine a separate code path","When accepting pluggable engines, capability-check optional methods before calling them","Prefer lf.collect(..., background=True) for engine-agnostic asynchronous execution","Keep a test that exercises each API entry point against every engine your app supports"],"tags":["polars","engine","async","not-implemented","plugin"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}