{"record":{"id":"529d7313865abd5c","repo":"pola-rs/polars","slug":"collect-all-is-not-supported-by-type-self-na","errorCode":null,"errorMessage":"`collect_all` is not supported by {type(self).__name__}","messagePattern":"`collect_all` is not supported by (.+?)","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/lazyframe/engine.py","lineNumber":211,"sourceCode":"    def collect_batches(\n        self,\n        lf: LazyFrame,\n        *,\n        optimizations: QueryOptFlags,\n        maintain_order: bool = True,\n        chunk_size: int | None = None,\n        lazy: bool = False,\n    ) -> Iterator[DataFrame]:\n        \"\"\"Execute `lf`, yielding its result in batches.\"\"\"\n        msg = f\"`collect_batches` is not supported by {type(self).__name__}\"\n        raise NotImplementedError(msg)\n\n    def collect_all(\n        self, lfs: Iterable[LazyFrame], *, optimizations: QueryOptFlags\n    ) -> list[DataFrame]:\n        \"\"\"Execute several queries, potentially in parallel.\"\"\"\n        msg = f\"`collect_all` is not supported by {type(self).__name__}\"\n        raise NotImplementedError(msg)\n\n    def collect_all_async(\n        self,\n        lfs: Iterable[LazyFrame],\n        *,\n        optimizations: QueryOptFlags,\n        gevent: bool = False,\n    ) -> AsyncResult[list[DataFrame]]:\n        \"\"\"Execute several queries asynchronously.\"\"\"\n        msg = f\"`collect_all_async` is not supported by {type(self).__name__}\"\n        raise NotImplementedError(msg)\n\n    def sink_parquet(\n        self,\n        lf: LazyFrame,\n        path: str | Path | IO[bytes] | PartitionBy,\n        *,\n        compression: ParquetCompression,","sourceCodeStart":193,"sourceCodeEnd":229,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/lazyframe/engine.py#L193-L229","documentation":"`Engine.collect_all` (engine.py:211) is the optional multi-query execution hook used by `pl.collect_all`; its default stub raises `NotImplementedError` naming the engine. It executes several `LazyFrame`s potentially in parallel and is implemented only by the local engine family. `RemoteEngine` and custom `Engine` subclasses that define just `collect`/`execute` hit the stub.","triggerScenarios":"`pl.collect_all([lf1, lf2], engine=pl.RemoteEngine())`, or `pl.collect_all(lfs, engine=my_engine)` with a minimal custom engine. Also occurs when a non-local engine is set as the global affinity and `pl.collect_all` inherits it.","commonSituations":"Fan-out ETL code that executes a list of built queries with one engine argument; switching a codebase from local to remote execution by only changing the engine; half-finished custom backends used across a shared utility function.","solutions":["Collect sequentially with the engine's supported path: `[lf.collect(engine=remote) for lf in lfs]`","Use a local engine: `pl.collect_all(lfs, engine='streaming')` (parallel multi-query works there)","Implement `collect_all` on your custom `Engine` subclass","Check engine capability before dispatching when the engine is configurable"],"exampleFix":"# before\ndfs = pl.collect_all([lf1, lf2], engine=pl.RemoteEngine())  # NotImplementedError\n\n# after\ndfs = [lf.collect(engine=remote) for lf in (lf1, lf2)]","handlingStrategy":"validation","validationCode":"from polars.lazyframe.engine import Engine\n\ndef supports_collect_all(engine: pl.Engine) -> bool:\n    return type(engine).collect_all is not Engine.collect_all\n\nif supports_collect_all(engine):\n    dfs = pl.collect_all(lfs, engine=engine)\nelse:\n    dfs = [lf.collect(engine=engine) for lf in lfs]","typeGuard":null,"tryCatchPattern":"try:\n    dfs = pl.collect_all(lfs, engine=engine)\nexcept NotImplementedError as e:\n    if 'collect_all' in str(e):\n        dfs = [lf.collect(engine=engine) for lf in lfs]\n    else:\n        raise","preventionTips":["Wrap multi-query dispatch in one helper that can fall back to sequential collect","Prefer local engines for pl.collect_all fan-outs","Check engine capabilities once at startup, not per call, in hot paths"],"tags":["polars","engine","multi-query","not-implemented","plugin"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}