pola-rs/polars · error · ValueError

Invalid engine argument {engine=}

Error message

Invalid engine argument {engine=}

What it means

`RemoteEngine.__init__` (engine_remote.py:126) validates its `engine` keyword — the preferred worker engine hint that also controls plan rendering — against the `EngineTypeName` literal `{'auto','in-memory','streaming','gpu'}`. Unlike the global engine-name resolver, the legacy alias `'cpu'` is NOT accepted here because validation uses `get_args(EngineTypeName)`. The check runs at construction time, before the `polars_cloud` dependency is imported.

Source

Thrown at py-polars/src/polars/lazyframe/engine_remote.py:128

    """Additional options forwarded to the distributed planner."""

    def __init__(
        self,
        context: pc.ClientContext | None = None,
        *,
        scaling_mode: ScalingMode = "auto",
        engine: EngineTypeName = "auto",
        plan_type: PlanTypePreference = "dot",
        n_retries: int = 0,
        labels: list[str] | str | None = None,
        **kwargs: Any,
    ) -> None:
        if scaling_mode not in _SCALING_MODES:
            msg = f"invalid `scaling_mode` {scaling_mode!r}"
            raise ValueError(msg)
        if engine not in _WORKER_ENGINE_NAMES:
            msg = f"Invalid engine argument {engine=}"
            raise ValueError(msg)
        if scaling_mode == "single-node" and kwargs:
            msg = (
                f"distributed options {sorted(kwargs)!r} are not supported with "
                "`scaling_mode='single-node'`"
            )
            raise ValueError(msg)

        # fail here rather than deep inside a sink
        import_optional(
            "polars_cloud",
            err_prefix="remote engine requested, but required package",
            install_message="Please install using the command `pip install polars-cloud`",
        )

        self.context = context
        self.scaling_mode = scaling_mode
        self.engine = engine
        self.plan_type = plan_type

View on GitHub (pinned to df599052da)

Solutions

1. Use `'auto'`, `'in-memory'`, `'streaming'`, or `'gpu'` for the RemoteEngine `engine` keyword
2. Replace legacy `'cpu'` with `'in-memory'` when targeting remote workers
3. Validate against `{'auto','in-memory','streaming','gpu'}` (not the global `SUPPORTED_ENGINE_NAMES` semantics) when the value is dynamic

Example fix

# before
engine = pl.RemoteEngine(ctx, engine='cpu')  # ValueError

# after
engine = pl.RemoteEngine(ctx, engine='in-memory')

# Stricter than the global engine registry: no 'cpu' alias here
REMOTE_WORKER_ENGINES = ('auto', 'in-memory', 'streaming', 'gpu')

def make_remote_engine(ctx, engine: str = 'auto', **kw):
    if engine not in REMOTE_WORKER_ENGINES:
        raise ValueError(f'worker engine must be one of {REMOTE_WORKER_ENGINES}, got {engine!r}')
    return pl.RemoteEngine(ctx, engine=engine, **kw)

from typing import TypeGuard

def is_worker_engine_name(value: object) -> TypeGuard[str]:
    return value in ('auto', 'in-memory', 'streaming', 'gpu')

try:
    engine = pl.RemoteEngine(ctx, engine=worker)
except ValueError as e:
    if 'Invalid engine argument' in str(e):
        engine = pl.RemoteEngine(ctx, engine='auto')
    else:
        raise

When it happens

Trigger: `pl.RemoteEngine(ctx, engine='cpu')` (legacy alias rejected here), `engine='remote'`, or any typo such as `'gpu-cuda'`/`'streaming-engine'`. Only the four literal names pass.

Common situations: Reusing a global engine-name string (that legitimately contains 'cpu') as the RemoteEngine worker hint; copy-pasting engine values between `lf.collect(engine=...)` and `RemoteEngine(engine=...)` without noticing the stricter set; config files shared across tools.

Related errors

• invalid `scaling_mode` {scaling_mode!r}
• distributed options {sorted(kwargs)!r} are not supported wit
• Invalid engine argument {engine=}
• `{name}` is not supported by the remote engine
• DataFrame `how` must be one of {{{allowed}}}, got {how!r}

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/b4beb07fecae3666. Report an issue: GitHub.