pola-rs/polars · error · ValueError

`{name}` is not supported by the remote engine

Error message

`{name}` is not supported by the remote engine

What it means

`RemoteEngine._reject_if_set` (engine_remote.py:225) enforces the option subset Polars Cloud actually supports: any sink option that is set to a truthy value gets `ValueError: '`name` is not supported by the remote engine'`. Per sink: parquet rejects `lazy`, `mkdir`, `sync_on_close`, `retries`, `sinked_paths_callback`; ipc additionally rejects `record_batch_size`, `_record_batch_statistics`, and `maintain_order=False` (passed as `not maintain_order`); csv additionally rejects any `compression` other than `'uncompressed'`, `compression_level`, `check_extension=False`, and `maintain_order=False`.

Source

Thrown at py-polars/src/polars/lazyframe/engine_remote.py:230

    def _sink_uri(path: Any) -> str | PartitionBy:
        """Validate a Polars Cloud sink target."""
        from polars.io.partition import PartitionBy

        if not isinstance(path, (str, PartitionBy)):
            msg = (
                "the remote engine can only sink to a URI or a `PartitionBy`, got "
                f"{qualified_type_name(path)!r}"
            )
            raise TypeError(msg)
        return path

    @staticmethod
    def _reject_if_set(**kwargs: Any) -> None:
        """Reject options unsupported by Polars Cloud."""
        for name, value in kwargs.items():
            if value:
                msg = f"`{name}` is not supported by the remote engine"
                raise ValueError(msg)

    def sink_parquet(
        self,
        lf: LazyFrame,
        path: str | Path | IO[bytes] | PartitionBy,
        *,
        compression: ParquetCompression,
        compression_level: int | None,
        statistics: bool | str | dict[str, bool],
        row_group_size: int | None,
        data_page_size: int | None,
        maintain_order: bool,
        storage_options: StorageOptionsDict | None,
        credential_provider: CredentialProviderFunction | Literal["auto"] | None,
        retries: int | None,
        sync_on_close: SyncOnCloseMethod | None,
        metadata: ParquetMetadata | None,
        arrow_schema: ArrowSchemaExportable | None,

View on GitHub (pinned to df599052da)

Solutions

1. Drop the unsupported option for remote sinks (defaults are the supported behavior)
2. For compressed CSV output, write uncompressed remotely then compress in a downstream step, or run locally: `lf.sink_csv(path, compression='gzip', engine='streaming')`
3. If you need `lazy=True`, `mkdir`, or `sync_on_close`, use a local engine for that sink
4. If `maintain_order` matters for ipc/csv remotely, note Polars Cloud only supports `maintain_order=True`

Example fix

# before
lf.sink_csv('s3://bucket/out.csv', engine=remote, compression='gzip')  # ValueError

# after
lf.sink_csv('s3://bucket/out.csv', engine=remote)  # uncompressed
# or compress locally:
lf.sink_csv('out.csv.gz', compression='gzip', engine='streaming')

REMOTE_FORBIDDEN = {
    'parquet': {'lazy', 'mkdir', 'sync_on_close', 'retries', 'sinked_paths_callback'},
    'ipc': {'lazy', 'mkdir', 'sync_on_close', 'retries', 'sinked_paths_callback',
            'record_batch_size', '_record_batch_statistics'},
    'csv': {'lazy', 'mkdir', 'sync_on_close', 'retries', 'compression_level',
            'check_extension'},
}

def check_remote_sink_kwargs(fmt: str, kwargs: dict) -> None:
    bad = [k for k in REMOTE_FORBIDDEN[fmt] if kwargs.get(k)]
    if fmt == 'csv' and kwargs.get('compression') not in (None, 'uncompressed'):
        bad.append('compression')
    if fmt in ('ipc', 'csv') and kwargs.get('maintain_order') is False:
        bad.append('maintain_order')
    if bad:
        raise ValueError(f'options not supported by the remote engine: {sorted(bad)}')

try:
    lf.sink_csv(uri, engine=remote, **sink_opts)
except ValueError as e:
    if 'not supported by the remote engine' in str(e):
        lf.sink_csv(uri, engine=remote)  # retry with defaults
    else:
        raise

When it happens

Trigger: `lf.sink_csv('s3://b/out.csv', engine=remote, compression='gzip')`; `lf.sink_parquet(uri, engine=remote, mkdir=True)`; `lf.sink_ipc(uri, engine=remote, record_batch_size=1000)`; any remote sink with `lazy=True` or `sync_on_close='data'` or `maintain_order=False` (ipc/csv).

Common situations: A sink helper with many keyword options reused for both local and remote engines; compressed CSV output required by a consumer while data must land in cloud storage; replicating local sink flags verbatim when moving to Polars Cloud.

Related errors

• invalid `scaling_mode` {scaling_mode!r}
• Invalid engine argument {engine=}
• distributed options {sorted(kwargs)!r} are not supported wit
• cannot apply delta_table_options for table of type {data_sou
• `sink_ndjson` is not supported by {type(self).__name__}

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/d69e1d3c4d82d634. Report an issue: GitHub.