pola-rs/polars · error · NotImplementedError

`sink_batches` is not supported by

Error message

`sink_batches` is not supported by {type(self).__name__}

What it means

`sink_batches` streams a LazyFrame's output to a user callback in batches, letting the caller control flushing/termination. Only engines with streaming sink support implement it; the base `Engine` raises NotImplementedError with this message otherwise.

Solutions

  1. Use `engine='streaming'`, which implements `sink_batches`.
  2. Use `collect_batches` (on a supporting engine) and invoke your callback per batch manually.
  3. Implement `sink_batches` in your custom Engine subclass.
  4. Check the engine's method support matrix in polars docs.

Example fix

// before
lf.sink_batches(lambda df: None, engine="custom")
// after
lf.sink_batches(lambda df: None, engine="streaming")
Defensive patterns

Strategy: try-catch

Validate before calling

if type(engine_obj).sink_batches is Engine.sink_batches:
    engine = "streaming"

Type guard

def supports_batch_sink(engine: object) -> bool:
    return type(engine).sink_batches is not Engine.sink_batches

Try / catch

try:
    lf.sink_batches(fn, engine=eng)
except NotImplementedError:
    for batch in lf.collect_batches():
        fn(batch)

Prevention

When it happens

Trigger: Calling `LazyFrame.sink_batches(lf, function, engine=<engine>)` where the engine does not override `sink_batches` (e.g. a custom engine).

Common situations: Custom batched write handlers (e.g. chunked uploads) used with a non-streaming or user-defined engine; porting streaming code between engines.

Related errors

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/31813e95cad02b62. Report an issue: GitHub.

Appendix: source

Thrown at py-polars/src/polars/lazyframe/engine.py:292

        mkdir: bool,
        optimizations: QueryOptFlags,
    ) -> None:
        """See :meth:`polars.LazyFrame.sink_ndjson`."""
        msg = f"`sink_ndjson` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)

    def sink_batches(
        self,
        lf: LazyFrame,
        function: Callable[[DataFrame], bool | None],
        *,
        chunk_size: int | None,
        maintain_order: bool,
        optimizations: QueryOptFlags,
    ) -> None:
        """See :meth:`polars.LazyFrame.sink_batches`."""
        msg = f"`sink_batches` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)


class _LocalEngine(Engine):
    """Base for in-process engines backed by `PyLazyFrame`."""

    _name: ClassVar[str]

    # Subclasses that do not call `super().__init__()` still see the default value.
    monitoring: bool | None = None
    """Whether queries are monitored (``None`` uses the configured default)."""

    def __init__(self, *, monitoring: bool | None = None) -> None:
        if monitoring:
            from polars._utils.monitoring import activate_monitoring

            activate_monitoring()
        self.monitoring = monitoring

View on GitHub (pinned to fe841f959e)