sgl-project/sglang · error · ValueError

experimental_sgl_marlin LoRA requires --lora-use-virtual-exp

Error message

experimental_sgl_marlin LoRA requires --lora-use-virtual-experts

What it means

Raised at MoE execution time by the experimental SGLang Marlin LoRA runner when the per-batch lora_info does not have lora_use_virtual_experts enabled together with max_lora_rank > 0. The experimental marlin LoRA kernels multiplex LoRA as 'virtual experts' inside the MoE routing, so the flag (and a positive rank) are prerequisites.

Source

Thrown at python/sglang/srt/lora/marlin_lora_temp/moe_runner.py:229

    from sglang.kernels.ops.moe.trtllm_lora_temp.virtual_experts import (
        merged_experts_fused_moe_lora_add,
    )
    from sglang.srt.layers.moe.token_dispatcher.standard import (
        StandardCombineInput,
        StandardDispatchOutput,
    )
    from sglang.srt.lora.trtllm_lora_temp import (
        get_lora_side_stream,
        is_two_stream_active,
    )
    from sglang.srt.lora.trtllm_lora_temp.environ import experimental_lora_enabled
    from sglang.srt.model_executor.runner import get_is_capture_mode

    if not isinstance(dispatch_output, StandardDispatchOutput):
        raise TypeError("experimental_sgl_marlin requires the standard MoE dispatcher")

    if not (lora_info.lora_use_virtual_experts and lora_info.max_lora_rank > 0):
        raise ValueError(
            "experimental_sgl_marlin LoRA requires --lora-use-virtual-experts"
        )

    hidden_states = dispatch_output.hidden_states
    topk_output = dispatch_output.topk_output
    topk_weights = topk_output.topk_weights
    topk_ids = topk_output.topk_ids
    if (
        topk_ids.ndim != 2
        or topk_weights.shape != topk_ids.shape
        or topk_ids.dtype != torch.int32
        or topk_weights.dtype != torch.float32
    ):
        raise ValueError(
            "experimental_sgl_marlin requires contiguous int32 ids and FP32 weights"
        )
    if not topk_ids.is_contiguous() or not topk_weights.is_contiguous():
        raise ValueError("experimental_sgl_marlin requires contiguous top-k tensors")

View on GitHub (pinned to 0132848349)

Solutions

1. Restart with --lora-use-virtual-experts and a valid --max-lora-rank > 0
2. Ensure the LoRA info passed to the fused_experts call carries the virtual-experts setting (see also the startup validation in policy.py)
3. If LoRA is not intended, disable the lora path so the stock Marlin fused path is used

Example fix

# before
--enable-lora --moe-runner-backend experimental_sgl_marlin
# after
--enable-lora --lora-use-virtual-experts --max-lora-rank 64 --moe-runner-backend experimental_sgl_marlin

assert lora_info.lora_use_virtual_experts and lora_info.max_lora_rank > 0, 'experimental_sgl_marlin LoRA needs --lora-use-virtual-experts'

When it happens

Trigger: dispatch_experimental_sgl_marlin_lora / _run_marlin_flow invoked with a lora_info where lora_use_virtual_experts is False or max_lora_rank is 0, typically because the server wasn't started with --lora-use-virtual-experts.

Common situations: Running the experimental sgl_marlin MoE path with LoRA enabled but virtual experts not turned on, or LoRA disabled at startup and enabled later without the flag.

Related errors

• experimental_sgl_marlin LoRA requires --lora-use-virtual-exp
• experimental_sgl_marlin EP requires --moe-a2a-backend none
• experimental_sgl_marlin LoRA requires --lora-backend triton
• experimental_sgl_marlin EP requires trivial expert placement
• Mixed shared-outer LoRA formats detected across loaded adapt

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1dde82fec57cf39b. Report an issue: GitHub.