{"record":{"id":"8f976f33c9437cad","repo":"sgl-project/sglang","slug":"sglang-rust-server-does-not-yet-apply-preferred","errorCode":null,"errorMessage":"SGLANG_RUST_SERVER does not yet apply --preferred-sampling-params (the Python TokenizerManager merges it into every request; the rust ingress has no equivalent). Launch without SGLANG_RUST_SERVER, or drop --preferred-sampling-params and send those values per request.","messagePattern":"SGLANG_RUST_SERVER does not yet apply --preferred-sampling-params \\(the Python TokenizerManager merges it into every request; the rust ingress has no equivalent\\)\\. Launch without SGLANG_RUST_SERVER, or drop --preferred-sampling-params and send those values per request\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/rust_server.py","lineNumber":395,"sourceCode":"        The caller gates this (``SGLANG_RUST_SERVER`` + rank 0); this always\n        creates.\n        \"\"\"\n        from sglang.srt.rust_extensions import load_rust_extension\n\n        Server = load_rust_extension(\"sglang.srt.rust_extensions._server\").Server\n\n        # Force turn off HF tokenizers rayon's unpinned global thread pool.\n        os.environ.setdefault(\"TOKENIZERS_PARALLELISM\", \"false\")\n\n        server_args = scheduler.server_args\n        # `TokenizerManager` merges these under each request's own sampling params\n        # (`{**preferred, **obj.sampling_params}`), and this server replaces that\n        # manager wholesale — so honouring the flag is not implemented here yet.\n        # Refuse rather than run: silently dropping it means generating with\n        # sampling the operator did not configure, and `/get_model_info` would go on\n        # advertising values no request ever receives.\n        if get_serving().preferred_sampling_params:\n            raise ValueError(\n                \"SGLANG_RUST_SERVER does not yet apply --preferred-sampling-params \"\n                \"(the Python TokenizerManager merges it into every request; the rust \"\n                \"ingress has no equivalent). Launch without SGLANG_RUST_SERVER, or \"\n                \"drop --preferred-sampling-params and send those values per request.\"\n            )\n        http_addr = f\"{get_serving().host}:{server_args.port}\"\n\n        # Per-DP-rank HTTP port with client load balancing. `None` when DP is off,\n        # so the rank is not conflated with rank 0 of a one-rank group.\n        dp_rank = scheduler.ps.attn_dp_rank if scheduler.ps.dp_size > 1 else None\n        if dp_rank is not None:\n            http_addr = f\"{get_serving().host}:{server_args.port + dp_rank}\"\n\n        launch_cores, server_cores = cls._partition_cores(\n            mm_workers=(\n                (server_args.mm_processor_worker_num or NativeMmHost.AUTO_MM_WORKERS)\n                if scheduler.model_config.is_multimodal\n                else 0","sourceCodeStart":377,"sourceCodeEnd":413,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/rust_server.py#L377-L413","documentation":"The experimental Rust HTTP ingress replaces the Python TokenizerManager, which is what merges --preferred-sampling-params into every request. Since the rust path has no equivalent, SGLang refuses to start rather than silently dropping server-configured sampling defaults.","triggerScenarios":"Launching with SGLANG_RUST_SERVER=1 together with --preferred-sampling-params (e.g. preferred_sampling_params JSON on the CLI/ServerArgs).","commonSituations":"Copy-pasting a Python-server launch command and adding the rust flag; benchmarking scripts carrying both options.","solutions":["Drop --preferred-sampling-params and send sampling values per request","Or unset SGLANG_RUST_SERVER and use the Python ingress"],"exampleFix":"# before\nSGLANG_RUST_SERVER=1 python -m sglang.launch_server --model ... --preferred-sampling-params '{\"temperature\":0.7}'\n# after\nSGLANG_RUST_SERVER=1 python -m sglang.launch_server --model ...","handlingStrategy":"validation","validationCode":"import os\nif os.environ.get('SGLANG_RUST_SERVER'):\n    assert not preferred_sampling_params, 'incompatible with rust ingress'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep launch scripts flag-compatibility aware","Send sampling params per request when using rust server"],"tags":["sglang","rust-server","config-conflict","startup"],"backgroundTag":"incompatible-launch-flags","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}