sgl-project/sglang · error · ValueError

fuse_swiglu_interleaved set on an incompatible fused_moe cal

Error message

fuse_swiglu_interleaved set on an incompatible fused_moe call

What it means

Internal validation in the Triton fused MoE path: the fuse_swiglu_interleaved fast path was requested but the current call's parameters don't meet its strict prerequisites (no bias, not quantized, no router-weight-on-input, no hooks, bf16, plus preceding conditions in the validator). It signals a config/model combination that the fused interleaved SwiGLU kernel cannot handle.

Source

Thrown at python/sglang/srt/layers/moe/moe_runner/triton_utils/fused_moe.py:119

    is_gated: bool,
    has_gemm1_modifiers: bool,
    has_bias: bool,
    is_quantized: bool,
    apply_router_weight_on_input: bool,
    has_hooks: bool,
    dtype: torch.dtype,
) -> None:
    if not (
        activation == "silu"
        and is_gated
        and not has_gemm1_modifiers
        and not has_bias
        and not is_quantized
        and not apply_router_weight_on_input
        and not has_hooks
        and dtype == torch.bfloat16
    ):
        raise ValueError(
            "fuse_swiglu_interleaved set on an incompatible fused_moe call"
        )


def _use_moe_sum_reduce_torch_compile(num_tokens: int) -> bool:
    return num_tokens <= 32 and not is_batch_invariant_mode_enabled()


@register_custom_op(mutates_args=["hidden_states"])
def inplace_fused_experts(
    hidden_states: torch.Tensor,
    w1: torch.Tensor,
    w2: torch.Tensor,
    topk_weights: torch.Tensor,
    topk_ids: torch.Tensor,
    b1: Optional[torch.Tensor] = None,
    b2: Optional[torch.Tensor] = None,
    activation: str = "silu",

View on GitHub (pinned to 0132848349)

Solutions

1. Disable the fused interleaved SwiGLU path (turn off the fuse_swiglu_interleaved flag / server arg) so the standard MoE path runs
2. Run with bfloat16 compute dtype instead of fp16/fp32
3. Remove the incompatible ingredient: use an unquantized checkpoint, or a checkpoint without expert bias / router-weight-on-input
4. If you believe this combination should be supported, file an issue; currently the kernel only supports the plain bf16 unbiased case

Example fix

# before
fused_experts(..., fuse_swiglu_interleaved=True, is_quantized=True)  # raises

# after
fused_experts(..., fuse_swiglu_interleaved=False)  # fall back to standard path

def can_fuse_swiglu_interleaved(experts) -> bool:
    return (
        not any(getattr(m, "bias", None) is not None for m in experts)
        and not getattr(experts, "quant_method", None)
        and experts.dtype == torch.bfloat16
    )

if not can_fuse_swiglu_interleaved(experts):
    fuse_swiglu_interleaved = False

When it happens

Trigger: Calling fused_experts / the Triton MoE runner with fuse_swiglu_interleaved=True while any of: the experts have bias, are quantized (fp8/etc.), apply_router_weight_on_input is set, MoE hooks are registered, or the compute dtype is not bfloat16. Typically arises from models like interleaved-SwiGLU MoE variants (e.g. Qwen3-Next style) combined with quantization or non-bf16 dtypes.

Common situations: Enabling --enable-fused-swiglu-interleaved (or a model defaulting to it) together with fp8/awq quantization, or running in fp16 instead of bf16, or a checkpoint whose experts carry biases.

Related errors

• num_token_non_padded must be an integer tensor, got {num_tok
• topk_ids must be int32, got {topk_ids.dtype}
• Unsupported ascend_dispatcher_output_dtype: {self.ascend_dis
• The Triton WNA16 MoE backend only supports symmetric INT4 gr
• sparse_attn_v4_paged_decode expects fp16/bf16 q, got {q.dtyp

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/97040dd819296421. Report an issue: GitHub.