pola-rs/polars · error · NotImplementedError

`sink_ndjson` is not supported by {type(self).__name__}

Error message

`sink_ndjson` is not supported by {type(self).__name__}

What it means

`Engine.sink_ndjson` (engine.py:327) is the optional `Engine` hook behind `LazyFrame.sink_ndjson`. Unlike the parquet/ipc/csv sinks, `RemoteEngine` does NOT override it — only the local `_LocalEngine` family does. So this error fires both for custom engines and, concretely, for the built-in remote engine: NDJSON output to Polars Cloud is unimplemented.

Source

Thrown at py-polars/src/polars/lazyframe/engine.py:327

        self,
        lf: LazyFrame,
        path: str | Path | IO[bytes] | IO[str] | PartitionBy,
        *,
        compression: Literal["uncompressed", "gzip", "zstd"],
        compression_level: int | None,
        check_extension: bool,
        maintain_order: bool,
        storage_options: StorageOptionsDict | None,
        credential_provider: CredentialProviderFunction | Literal["auto"] | None,
        retries: int | None,
        sync_on_close: SyncOnCloseMethod | None,
        mkdir: bool,
        lazy: bool,
        optimizations: QueryOptFlags,
    ) -> LazyFrame | None:
        """See :meth:`polars.LazyFrame.sink_ndjson`."""
        msg = f"`sink_ndjson` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)

    def sink_batches(
        self,
        lf: LazyFrame,
        function: Callable[[DataFrame], bool | None],
        *,
        chunk_size: int | None,
        maintain_order: bool,
        lazy: bool,
        optimizations: QueryOptFlags,
    ) -> LazyFrame | None:
        """See :meth:`polars.LazyFrame.sink_batches`."""
        msg = f"`sink_batches` is not supported by {type(self).__name__}"
        raise NotImplementedError(msg)


class _LocalEngine(Engine):
    """Base for in-process engines backed by `PyLazyFrame`."""

View on GitHub (pinned to df599052da)

Solutions

1. Run the sink locally: `lf.sink_ndjson(path, engine='streaming')`
2. For remote execution, sink to a format RemoteEngine supports (parquet, ipc, csv) and convert afterwards
3. Collect remotely then write locally: `lf.collect(engine=remote).write_ndjson(path)`
4. Implement `sink_ndjson` on your custom `Engine` subclass

Example fix

# before
lf.sink_ndjson('s3://bucket/out.ndjson', engine=pl.RemoteEngine())  # NotImplementedError

# after
lf.sink_parquet('s3://bucket/out.parquet', engine=pl.RemoteEngine())
# or locally:
lf.sink_ndjson('out.ndjson', engine='streaming')

from polars.lazyframe.engine import Engine

def engine_can_sink_ndjson(engine: pl.Engine) -> bool:
    return type(engine).sink_ndjson is not Engine.sink_ndjson

remote = pl.RemoteEngine()
assert engine_can_sink_ndjson(remote) is False  # RemoteEngine lacks ndjson sinks
assert engine_can_sink_ndjson(pl.StreamingEngine()) is True

try:
    lf.sink_ndjson(uri, engine=engine)
except NotImplementedError as e:
    if 'sink_ndjson' in str(e):
        lf.sink_ndjson(uri, engine='streaming')  # local fallback
    else:
        raise

When it happens

Trigger: `lf.sink_ndjson('s3://bucket/out.ndjson', engine=pl.RemoteEngine())` raises '`sink_ndjson` is not supported by RemoteEngine'; likewise `lf.sink_ndjson(path, engine=my_engine)` for any custom `Engine` subclass lacking the override.

Common situations: Porting a local NDJSON export pipeline to Polars Cloud assuming format parity with sink_parquet; JSON-lines output required by downstream consumers (event streams, log pipelines) in a distributed setup; custom backends that never implemented NDJSON.

Related errors

• `sink_batches` is not supported by {type(self).__name__}
• `sink_parquet` is not supported by {type(self).__name__}
• `sink_ipc` is not supported by {type(self).__name__}
• `sink_csv` is not supported by {type(self).__name__}
• `collect_async` is not supported by {type(self).__name__}

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/ba99001bf756a46e. Report an issue: GitHub.