{"record":{"id":"b0a8d954b5d90517","repo":"sgl-project/sglang","slug":"cutedsl-masked-moe-supports-activation-silu-gat","errorCode":null,"errorMessage":"CuteDSL masked MoE supports activation 'silu' (gated) or 'relu2' (non-gated), got {activation!r}.","messagePattern":"CuteDSL masked MoE supports activation 'silu' \\(gated\\) or 'relu2' \\(non-gated\\), got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/moe/flashinfer_cutedsl_moe.py","lineNumber":112,"sourceCode":"        assert (\n            input_global_scale.dtype == torch.float32\n        ), f\"input_global_scale must be float32, got {input_global_scale.dtype}\"\n        assert input_global_scale.shape == (\n            num_experts,\n        ), f\"input_global_scale must be (l,), got {input_global_scale.shape}\"\n\n        a_q, a_q_sf = scaled_fp4_grouped_quantize(\n            hidden_states[0],\n            masked_m,\n            input_global_scale,\n        )\n\n    if activation == \"silu\":\n        gated = True\n    elif activation == \"relu2\":\n        gated = False\n    else:\n        raise ValueError(\n            f\"CuteDSL masked MoE supports activation 'silu' (gated) or \"\n            f\"'relu2' (non-gated), got {activation!r}.\"\n        )\n    # Gated (silu_and_mul) GEMM1 emits [gate, up] so w1 has 2*n rows; non-gated\n    # relu2 emits a single projection of n rows.\n    gemm1_out_dim = 2 * n if gated else n\n    assert (\n        w1.shape[-2] == gemm1_out_dim\n    ), f\"w1 last-2 dim must be {gemm1_out_dim} (gated={gated}), got {w1.shape}\"\n    assert (\n        w1.shape[-1] * 2 == k\n    ), f\"w1 last dim * 2 must equal k, got {w1.shape[-1]} vs k={k}\"\n    assert w2.shape[-2:] == (\n        k,\n        n // 2,\n    ), f\"w2 shape mismatch, got {w2.shape[-2:]}, expected {(k, n//2)}\"\n    assert w1_alpha.shape == (\n        num_experts,","sourceCodeStart":94,"sourceCodeEnd":130,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/flashinfer_cutedsl_moe.py#L94-L130","documentation":"flashinfer_cutedsl_moe_masked only supports activation='silu' (gated MoE, w1 has 2*n rows) or 'relu2' (non-gated, single n-row projection). Any other activation string is rejected before building the GEMM shapes.","triggerScenarios":"Passing activation='gelu', 'gelu_sigmoid' (GeGLU), 'silu_and_mul', or any string other than the two supported ones; typically propagated from a model config's hidden_act field.","commonSituations":"Wiring a new model whose config.hidden_act is e.g. gelu_new into the FlashInfer CuteDSL MoE path without an activation translation layer.","solutions":["Map your model's activation to 'silu' or 'relu2' if semantically equivalent; otherwise route the model to a different MoE backend (e.g. triton fused MoE)","Check for a typo: pass the exact strings 'silu' or 'relu2'","If the model genuinely needs gelu, add support in the wrapper or disable the cutedsl path for that model"],"exampleFix":"// before\nflashinfer_cutedsl_moe_masked(..., activation='gelu_sigmoid')\n// after\nflashinfer_cutedsl_moe_masked(..., activation='silu')  # if model uses SiLU-gating\n# or route model to triton backend for unsupported activations","handlingStrategy":"validation","validationCode":"if activation not in ('silu', 'relu2'):\n    raise ValueError(f'model activation {activation!r} unsupported by cutedsl MoE; use triton backend')","typeGuard":"def cutedsl_supported_activation(a: str) -> bool:\n    return a in ('silu', 'relu2')","tryCatchPattern":null,"preventionTips":["Map model hidden_act to the supported set when wiring new models","Add a startup assertion on activation before selecting the cutedsl backend"],"tags":["flashinfer","cutedsl","moe","activation","unsupported-config"],"backgroundTag":"unsupported-activation-function","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}