{"record":{"id":"97040dd819296421","repo":"sgl-project/sglang","slug":"fuse-swiglu-interleaved-set-on-an-incompatible-fus","errorCode":null,"errorMessage":"fuse_swiglu_interleaved set on an incompatible fused_moe call","messagePattern":"fuse_swiglu_interleaved set on an incompatible fused_moe call","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/moe/moe_runner/triton_utils/fused_moe.py","lineNumber":119,"sourceCode":"    is_gated: bool,\n    has_gemm1_modifiers: bool,\n    has_bias: bool,\n    is_quantized: bool,\n    apply_router_weight_on_input: bool,\n    has_hooks: bool,\n    dtype: torch.dtype,\n) -> None:\n    if not (\n        activation == \"silu\"\n        and is_gated\n        and not has_gemm1_modifiers\n        and not has_bias\n        and not is_quantized\n        and not apply_router_weight_on_input\n        and not has_hooks\n        and dtype == torch.bfloat16\n    ):\n        raise ValueError(\n            \"fuse_swiglu_interleaved set on an incompatible fused_moe call\"\n        )\n\n\ndef _use_moe_sum_reduce_torch_compile(num_tokens: int) -> bool:\n    return num_tokens <= 32 and not is_batch_invariant_mode_enabled()\n\n\n@register_custom_op(mutates_args=[\"hidden_states\"])\ndef inplace_fused_experts(\n    hidden_states: torch.Tensor,\n    w1: torch.Tensor,\n    w2: torch.Tensor,\n    topk_weights: torch.Tensor,\n    topk_ids: torch.Tensor,\n    b1: Optional[torch.Tensor] = None,\n    b2: Optional[torch.Tensor] = None,\n    activation: str = \"silu\",","sourceCodeStart":101,"sourceCodeEnd":137,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/moe_runner/triton_utils/fused_moe.py#L101-L137","documentation":"Internal validation in the Triton fused MoE path: the fuse_swiglu_interleaved fast path was requested but the current call's parameters don't meet its strict prerequisites (no bias, not quantized, no router-weight-on-input, no hooks, bf16, plus preceding conditions in the validator). It signals a config/model combination that the fused interleaved SwiGLU kernel cannot handle.","triggerScenarios":"Calling fused_experts / the Triton MoE runner with fuse_swiglu_interleaved=True while any of: the experts have bias, are quantized (fp8/etc.), apply_router_weight_on_input is set, MoE hooks are registered, or the compute dtype is not bfloat16. Typically arises from models like interleaved-SwiGLU MoE variants (e.g. Qwen3-Next style) combined with quantization or non-bf16 dtypes.","commonSituations":"Enabling --enable-fused-swiglu-interleaved (or a model defaulting to it) together with fp8/awq quantization, or running in fp16 instead of bf16, or a checkpoint whose experts carry biases.","solutions":["Disable the fused interleaved SwiGLU path (turn off the fuse_swiglu_interleaved flag / server arg) so the standard MoE path runs","Run with bfloat16 compute dtype instead of fp16/fp32","Remove the incompatible ingredient: use an unquantized checkpoint, or a checkpoint without expert bias / router-weight-on-input","If you believe this combination should be supported, file an issue; currently the kernel only supports the plain bf16 unbiased case"],"exampleFix":"# before\nfused_experts(..., fuse_swiglu_interleaved=True, is_quantized=True)  # raises\n\n# after\nfused_experts(..., fuse_swiglu_interleaved=False)  # fall back to standard path","handlingStrategy":"validation","validationCode":"def can_fuse_swiglu_interleaved(experts) -> bool:\n    return (\n        not any(getattr(m, \"bias\", None) is not None for m in experts)\n        and not getattr(experts, \"quant_method\", None)\n        and experts.dtype == torch.bfloat16\n    )\n\nif not can_fuse_swiglu_interleaved(experts):\n    fuse_swiglu_interleaved = False","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat fuse_swiglu_interleaved as bf16-unquantized-only until upstream expands support","Log the flag at startup so quantized runs make the incompatibility obvious"],"tags":["moe","triton","swiglu","quantization","dtype","feature-guard"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}