pola-rs/polars · error · NotImplementedError

`sink_parquet` is not supported by {type(self).__name__}

Error message

`sink_parquet` is not supported by {type(self).__name__}

What it means

`Engine.sink_parquet` (engine.py:248) is the optional hook behind `LazyFrame.sink_parquet` for writing a query result to Parquet without materializing it in memory. Its default `Engine` stub raises `NotImplementedError` naming the engine. All built-in engines implement it — `_LocalEngine` subclasses directly and `RemoteEngine` via Polars Cloud — so in practice this error comes from custom `Engine` subclasses that only implement the abstract `collect`/`execute`.

Source

Thrown at py-polars/src/polars/lazyframe/engine.py:248

        compression_level: int | None,
        statistics: bool | str | dict[str, bool],
        row_group_size: int | None,
        data_page_size: int | None,
        maintain_order: bool,
        storage_options: StorageOptionsDict | None,
        credential_provider: CredentialProviderFunction | Literal["auto"] | None,
        retries: int | None,
        sync_on_close: SyncOnCloseMethod | None,
        metadata: ParquetMetadata | None,
        arrow_schema: ArrowSchemaExportable | None,
        mkdir: bool,
        lazy: bool,
        optimizations: QueryOptFlags,
        sinked_paths_callback: SinkedPathsCallback | None,
    ) -> LazyFrame | None:
        """See :meth:`polars.LazyFrame.sink_parquet`."""
        msg = f"`sink_parquet` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)

    def sink_ipc(
        self,
        lf: LazyFrame,
        path: str | Path | IO[bytes] | PartitionBy,
        *,
        compression: IpcCompression | None,
        compat_level: CompatLevel | None,
        record_batch_size: int | None,
        maintain_order: bool,
        storage_options: StorageOptionsDict | None,
        credential_provider: CredentialProviderFunction | Literal["auto"] | None,
        retries: int | None,
        sync_on_close: SyncOnCloseMethod | None,
        mkdir: bool,
        lazy: bool,
        optimizations: QueryOptFlags,
        _record_batch_statistics: bool,

View on GitHub (pinned to df599052da)

Solutions

1. Use a built-in engine for sinks: `lf.sink_parquet(path, engine='streaming')` (default) or `engine=pl.RemoteEngine()` for cloud URIs
2. Implement `sink_parquet(self, lf, path, *, ...)` on your `Engine` subclass with the exact keyword signature shown in the stub
3. Collect then write eagerly as a stopgap: `lf.collect().write_parquet(path)`
4. Check the engine supports sinks before dispatch (see defense) when engines are pluggable at runtime

Example fix

# before
class MyEngine(pl.Engine):
    @property
    def name(self): return 'my'
    def collect(self, lf, *, optimizations, background=False, post_opt_callback=None): ...
    def execute(self, lf, *, optimizations): ...
lf.sink_parquet('out.parquet', engine=MyEngine())  # NotImplementedError

# after: add the override
class MyEngine(pl.Engine):
    ...
    def sink_parquet(self, lf, path, *, compression, compression_level, statistics,
                     row_group_size, data_page_size, maintain_order, storage_options,
                     credential_provider, retries, sync_on_close, metadata, arrow_schema,
                     mkdir, lazy, optimizations, sinked_paths_callback):
        ...  # write lf's result to path

from polars.lazyframe.engine import Engine

def engine_can_sink(engine: pl.Engine, fmt: str = 'sink_parquet') -> bool:
    return getattr(type(engine), fmt) is not getattr(Engine, fmt)

if not engine_can_sink(my_engine, 'sink_parquet'):
    my_engine = pl.StreamingEngine()  # or collect+write_parquet fallback

try:
    lf.sink_parquet(path, engine=my_engine)
except NotImplementedError as e:
    if 'sink_parquet' in str(e):
        lf.collect().write_parquet(path)  # eager fallback
    else:
        raise

When it happens

Trigger: `lf.sink_parquet(path, engine=my_engine)` where `my_engine` is a user-defined `pl.Engine` subclass without a `sink_parquet` override. Commonly a stub/test engine or an in-progress alternative backend.

Common situations: Writing a custom execution backend (e.g. pushing the plan to another query engine) and testing the full `LazyFrame` API surface; injecting a recording engine in tests that only implements collect; copying a minimal engine example from the docs and expecting all sink methods to work.

Related errors

• `sink_ipc` is not supported by {type(self).__name__}
• `sink_csv` is not supported by {type(self).__name__}
• `collect_async` is not supported by {type(self).__name__}
• `collect_batches` is not supported by {type(self).__name__}
• `collect_all` is not supported by {type(self).__name__}

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/480c204722e63190. Report an issue: GitHub.