{"record":{"id":"450740293ef645e2","repo":"sgl-project/sglang","slug":"unsupported-activation-activation-type-450740","errorCode":null,"errorMessage":"Unsupported activation: {activation_type}","messagePattern":"Unsupported activation: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/moe.py","lineNumber":578,"sourceCode":"    gateup_output: torch.Tensor,\n    topk_weights: torch.Tensor | None = None,\n    use_interleaved: bool = True,\n):\n    if activation_type == \"silu_and_mul\":\n        assert (\n            gateup_output.is_contiguous()\n        ), f\"{gateup_output.shape=} {gateup_output.stride()=}\"\n        assert gateup_output.ndim == 2, f\"{gateup_output.shape=}\"\n        out_dtype = None\n        if gateup_output.numel() == 0:\n            return gateup_output.new_zeros(\n                *gateup_output.shape[:-1], gateup_output.shape[-1] // 2, dtype=out_dtype\n            )\n\n        return silu_and_mul(\n            gateup_output, topk_weights, out_dtype, use_interleaved=use_interleaved\n        )\n    raise ValueError(f\"Unsupported activation: {activation_type}\")\n\n\ndef moe_tp_forward(\n    hidden_states: torch.Tensor,\n    topk_weights: torch.Tensor,\n    topk_ids: torch.Tensor,\n    w13_weight_E_2f_D: torch.Tensor,\n    w2_weight_EDf: torch.Tensor,\n    w13_bias_E_2f: torch.Tensor | None = None,\n    w2_bias_ED: torch.Tensor | None = None,\n    activation_type: str = \"silu_and_mul\",\n    use_interleaved: bool = True,\n) -> torch.Tensor:\n    orig_shape: torch.Size = hidden_states.shape\n    hidden_states_TD, topk_weights_TK, topk_ids_TK, top_k, num_experts = (\n        make_forward_inputs_2d(hidden_states, topk_weights, topk_ids, w2_weight_EDf)\n    )\n    del hidden_states, topk_weights, topk_ids","sourceCodeStart":560,"sourceCodeEnd":596,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/moe.py#L560-L596","documentation":"The activation() helper in inkling_common/moe.py only supports the recognized activation types (falling through to silu_and_mul); any other activation_type string reaches the final raise. It is hit from apply_group_norm_silu, forward, the eager reference resblock, and moe_tp_forward.","triggerScenarios":"Passing an activation_type value not handled by the if/elif chain (e.g. a typo like 'silu-mul', 'gelu', or an unsupported enum) into activation().","commonSituations":"Model config advertises a non-silu activation; custom model variant wired with a new activation name not yet added to the dispatcher.","solutions":["Check the activation_type string actually passed and correct it to a supported value (silu-family)","If a new activation is genuinely required, extend the dispatcher in activation() with a branch and kernel support","Verify the model config's hidden_act matches what the Inkling MoE path supports"],"exampleFix":"# before\nout = activation(gateup, topk_weights, activation_type='relu')\n# after\nout = activation(gateup, topk_weights, activation_type='silu')","handlingStrategy":"validation","validationCode":"SUPPORTED = {'silu'}  # per dispatcher\nassert activation_type in SUPPORTED, f'unsupported {activation_type}'","typeGuard":"def is_supported_activation(name: str) -> bool:\n    return name in {'silu'}","tryCatchPattern":"try:\n    out = activation(...)\nexcept ValueError as e:\n    if 'Unsupported activation' in str(e): raise ConfigError(...) from e\n    raise","preventionTips":["Centralize allowed activation names in a constant shared with config parsing"],"tags":["activation","moe","config-mismatch","sglang"],"backgroundTag":"unsupported-enum-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}