{"record":{"id":"f08451fd1d531a12","repo":"sgl-project/sglang","slug":"unsupported-activation-activation-type","errorCode":null,"errorMessage":"Unsupported activation: {ACTIVATION_TYPE}","messagePattern":"Unsupported activation: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/moe/fused_moe_triton_kernels.py","lineNumber":1079,"sourceCode":"def _apply_activation(x, ACTIVATION_TYPE: tl.constexpr):\n    \"\"\"\n    Apply activation function based on compile-time constant.\n\n    Args:\n        x: Input tensor (converted to float32 inside)\n        ACTIVATION_TYPE: Compile-time constant string (\"silu\" or \"gelu\")\n\n    Returns:\n        Activated output in the same dtype as input\n    \"\"\"\n    x = x.to(tl.float32)\n    if ACTIVATION_TYPE == \"silu\":\n        return x * tl.sigmoid(x)\n    elif ACTIVATION_TYPE == \"gelu\":\n        kAlpha = 0.7978845608028654\n        return 0.5 * x * (1 + tanh(kAlpha * (x + 0.044715 * x * x * x)))\n    else:\n        raise ValueError(f\"Unsupported activation: {ACTIVATION_TYPE}\")\n\n\n@triton.jit\ndef act_and_mul_kernel(\n    gateup_output,\n    down_input,\n    hidden_size,\n    expert_ids_ptr,\n    expert_step: tl.constexpr,\n    BLOCK_SIZE: tl.constexpr,\n    ACTIVATION_TYPE: tl.constexpr,\n    SWIGLU_LIMIT: tl.constexpr = 0.0,\n    HAS_SWIGLU_LIMIT: tl.constexpr = False,\n    HAS_EXPERT_FILTER: tl.constexpr = True,\n):\n    \"\"\"\n    Unified activation and multiply kernel that handles both sorted and unsorted routing,\n    and both SiLU and GELU activations using compile-time constants.","sourceCodeStart":1061,"sourceCodeEnd":1097,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/moe/fused_moe_triton_kernels.py#L1061-L1097","documentation":"ValueError raised at runtime inside the Triton-compiled _apply_activation function. The activation is baked in as the compile-time constant ACTIVATION_TYPE; only 'silu' and 'gelu' branches exist, and anything else hits the else branch raising this error.","triggerScenarios":"Passing an activation string other than \"silu\" or \"gelu\" (e.g. \"gelu_tanh\", \"relu\", \"swiglu\") to act_and_mul_kernel, which forwards it to _apply_activation as a constexpr.","commonSituations":"Adding a new gated-MLP activation to a model without extending this kernel; case/typo mistakes (\"SiLU\", \"gelu-and\"); a model config carrying an activation name the kernel never implemented.","solutions":["Fix the activation string to exactly \"silu\" or \"gelu\" if one of those was intended","If a new activation is genuinely required, add an elif branch to _apply_activation in fused_moe_triton_kernels.py","Map nonstandard config names (e.g. gelu_new) to \"gelu\" at the caller before invoking the kernel"],"exampleFix":"// before\nact_and_mul_kernel[grid](out, inp, \"gelu_tanh\", N, ...)\n// after\n# gelu_tanh ≈ the tanh approximation already implemented as \"gelu\"\nact_and_mul_kernel[grid](out, inp, \"gelu\", N, ...)","handlingStrategy":"validation","validationCode":"assert activation in (\"silu\", \"gelu\"), f\"unsupported activation {activation}\"","typeGuard":"def is_supported_activation(name: str) -> bool:\n    return name in (\"silu\", \"gelu\")","tryCatchPattern":"try:\n    act_and_mul_kernel[grid](out, inp, act, N)\nexcept ValueError as e:\n    if \"Unsupported activation\" in str(e):\n        raise ValueError(f\"model config requests {act!r}; kernel supports silu/gelu\") from e\n    raise","preventionTips":["Whitelist activation names at model-load time","Map config aliases (gelu_new, gelu_tanh -> gelu) before kernel dispatch"],"tags":["moe","triton","activation","unsupported-operation"],"backgroundTag":"unsupported-activation-function","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}