{"record":{"id":"d402609a8876ef9c","repo":"pola-rs/polars","slug":"distributed-options-sorted-kwargs-r-are-not-sup","errorCode":null,"errorMessage":"distributed options {sorted(kwargs)!r} are not supported with `scaling_mode='single-node'`","messagePattern":"distributed options (.+?) are not supported with `scaling_mode='single-node'`","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"py-polars/src/polars/lazyframe/engine_remote.py","lineNumber":134,"sourceCode":"        scaling_mode: ScalingMode = \"auto\",\n        engine: EngineTypeName = \"auto\",\n        plan_type: PlanTypePreference = \"dot\",\n        n_retries: int = 0,\n        labels: list[str] | str | None = None,\n        **kwargs: Any,\n    ) -> None:\n        if scaling_mode not in _SCALING_MODES:\n            msg = f\"invalid `scaling_mode` {scaling_mode!r}\"\n            raise ValueError(msg)\n        if engine not in _WORKER_ENGINE_NAMES:\n            msg = f\"Invalid engine argument {engine=}\"\n            raise ValueError(msg)\n        if scaling_mode == \"single-node\" and kwargs:\n            msg = (\n                f\"distributed options {sorted(kwargs)!r} are not supported with \"\n                \"`scaling_mode='single-node'`\"\n            )\n            raise ValueError(msg)\n\n        # fail here rather than deep inside a sink\n        import_optional(\n            \"polars_cloud\",\n            err_prefix=\"remote engine requested, but required package\",\n            install_message=\"Please install using the command `pip install polars-cloud`\",\n        )\n\n        self.context = context\n        self.scaling_mode = scaling_mode\n        self.engine = engine\n        self.plan_type = plan_type\n        self.n_retries = n_retries\n        self.labels = [labels] if isinstance(labels, str) else labels\n        self.config = kwargs\n\n    @property\n    def name(self) -> str:","sourceCodeStart":116,"sourceCodeEnd":152,"githubUrl":"https://github.com/pola-rs/polars/blob/df599052daf96e7a9cc30a3b0c6bd25d6947e3c0/py-polars/src/polars/lazyframe/engine_remote.py#L116-L152","documentation":"`RemoteEngine.__init__` (engine_remote.py:129) rejects any extra keyword arguments when `scaling_mode='single-node'`, because those kwargs (`max_workers`, `min_workers`, `shuffle_format`, `partitions_per_worker`, ...) are forwarded to the distributed planner and are meaningless for single-node runs. The message lists the offending option names sorted, and the check runs at construction time ('fail here rather than deep inside a sink').","triggerScenarios":"`pl.RemoteEngine(ctx, scaling_mode='single-node', max_workers=8)`, or any other distributed-planner kwarg combined with `scaling_mode='single-node'`. The kwargs are collected via `**kwargs`, so any unrecognized keyword silently becomes a distributed option and trips this check.","commonSituations":"A shared engine-construction helper that always passes cluster sizing options, called once with 'single-node' for a small job; typos in option names becoming unintended `**kwargs`; flipping scaling mode in config while leaving sizing options in place.","solutions":["Remove the distributed options when using `scaling_mode='single-node'`","Or switch to `scaling_mode='distributed'` (or `'auto'`) if the options should apply","Split engine construction into two presets (single-node vs distributed) instead of one parameterized call"],"exampleFix":"# before\nengine = pl.RemoteEngine(ctx, scaling_mode='single-node', max_workers=8)  # ValueError\n\n# after\nengine = pl.RemoteEngine(ctx, scaling_mode='distributed', max_workers=8)\n# or, for one node:\nengine = pl.RemoteEngine(ctx, scaling_mode='single-node')","handlingStrategy":"validation","validationCode":"DISTRIBUTED_OPTIONS = {'max_workers', 'min_workers', 'shuffle_format', 'partitions_per_worker'}\n\ndef make_remote_engine(ctx, scaling_mode='auto', **opts):\n    if scaling_mode == 'single-node' and opts:\n        raise ValueError(\n            f'options {sorted(opts)} require scaling_mode=\\'distributed\\', not \\'single-node\\''\n        )\n    return pl.RemoteEngine(ctx, scaling_mode=scaling_mode, **opts)","typeGuard":null,"tryCatchPattern":"try:\n    engine = pl.RemoteEngine(ctx, scaling_mode=mode, **cluster_opts)\nexcept ValueError as e:\n    if 'single-node' in str(e):\n        engine = pl.RemoteEngine(ctx, scaling_mode='distributed', **cluster_opts)\n    else:\n        raise","preventionTips":["Build two engine presets: a bare single-node one and a distributed one with sizing options","Validate scaling_mode together with kwargs at your own config layer","Never forward unvalidated **kwargs into RemoteEngine; unknown keywords become distributed options"],"tags":["polars","remote","cloud","distributed","config","valueerror"],"backgroundTag":null,"analyzedSha":"df599052daf96e7a9cc30a3b0c6bd25d6947e3c0","analyzedAt":"2026-08-16T12:10:03.978Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}