{"record":{"id":"1dde82fec57cf39b","repo":"sgl-project/sglang","slug":"experimental-sgl-marlin-lora-requires-lora-use-v","errorCode":null,"errorMessage":"experimental_sgl_marlin LoRA requires --lora-use-virtual-experts","messagePattern":"experimental_sgl_marlin LoRA requires --lora-use-virtual-experts","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/lora/marlin_lora_temp/moe_runner.py","lineNumber":229,"sourceCode":"    from sglang.kernels.ops.moe.trtllm_lora_temp.virtual_experts import (\n        merged_experts_fused_moe_lora_add,\n    )\n    from sglang.srt.layers.moe.token_dispatcher.standard import (\n        StandardCombineInput,\n        StandardDispatchOutput,\n    )\n    from sglang.srt.lora.trtllm_lora_temp import (\n        get_lora_side_stream,\n        is_two_stream_active,\n    )\n    from sglang.srt.lora.trtllm_lora_temp.environ import experimental_lora_enabled\n    from sglang.srt.model_executor.runner import get_is_capture_mode\n\n    if not isinstance(dispatch_output, StandardDispatchOutput):\n        raise TypeError(\"experimental_sgl_marlin requires the standard MoE dispatcher\")\n\n    if not (lora_info.lora_use_virtual_experts and lora_info.max_lora_rank > 0):\n        raise ValueError(\n            \"experimental_sgl_marlin LoRA requires --lora-use-virtual-experts\"\n        )\n\n    hidden_states = dispatch_output.hidden_states\n    topk_output = dispatch_output.topk_output\n    topk_weights = topk_output.topk_weights\n    topk_ids = topk_output.topk_ids\n    if (\n        topk_ids.ndim != 2\n        or topk_weights.shape != topk_ids.shape\n        or topk_ids.dtype != torch.int32\n        or topk_weights.dtype != torch.float32\n    ):\n        raise ValueError(\n            \"experimental_sgl_marlin requires contiguous int32 ids and FP32 weights\"\n        )\n    if not topk_ids.is_contiguous() or not topk_weights.is_contiguous():\n        raise ValueError(\"experimental_sgl_marlin requires contiguous top-k tensors\")","sourceCodeStart":211,"sourceCodeEnd":247,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/lora/marlin_lora_temp/moe_runner.py#L211-L247","documentation":"Raised at MoE execution time by the experimental SGLang Marlin LoRA runner when the per-batch lora_info does not have lora_use_virtual_experts enabled together with max_lora_rank > 0. The experimental marlin LoRA kernels multiplex LoRA as 'virtual experts' inside the MoE routing, so the flag (and a positive rank) are prerequisites.","triggerScenarios":"dispatch_experimental_sgl_marlin_lora / _run_marlin_flow invoked with a lora_info where lora_use_virtual_experts is False or max_lora_rank is 0, typically because the server wasn't started with --lora-use-virtual-experts.","commonSituations":"Running the experimental sgl_marlin MoE path with LoRA enabled but virtual experts not turned on, or LoRA disabled at startup and enabled later without the flag.","solutions":["Restart with --lora-use-virtual-experts and a valid --max-lora-rank > 0","Ensure the LoRA info passed to the fused_experts call carries the virtual-experts setting (see also the startup validation in policy.py)","If LoRA is not intended, disable the lora path so the stock Marlin fused path is used"],"exampleFix":"# before\n--enable-lora --moe-runner-backend experimental_sgl_marlin\n# after\n--enable-lora --lora-use-virtual-experts --max-lora-rank 64 --moe-runner-backend experimental_sgl_marlin","handlingStrategy":"validation","validationCode":"assert lora_info.lora_use_virtual_experts and lora_info.max_lora_rank > 0, 'experimental_sgl_marlin LoRA needs --lora-use-virtual-experts'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always launch the experimental marlin runner with --lora-use-virtual-experts and --max-lora-rank > 0","Rely on the startup-time policy validation to catch this before serving"],"tags":["lora","marlin","moe","virtual-experts","experimental","sglang"],"backgroundTag":"missing-feature-flag","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}