{"record":{"id":"876c1fd97ca7f7d9","repo":"sgl-project/sglang","slug":"unsupported-activation-hidden-act-only-silu-is-876c1f","errorCode":null,"errorMessage":"Unsupported activation: {hidden_act}. Only silu is supported for now.","messagePattern":"Unsupported activation: (.+?)\\. Only silu is supported for now\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/step3_vl.py","lineNumber":96,"sourceCode":"        prefix: str = \"\",\n    ) -> None:\n        super().__init__()\n        self.gate_up_proj = MergedColumnParallelLinear(\n            hidden_size,\n            [intermediate_size] * 2,\n            bias=False,\n            quant_config=quant_config,\n            prefix=add_prefix(\"gate_up_proj\", prefix),\n        )\n        self.down_proj = RowParallelLinear(\n            intermediate_size,\n            hidden_size,\n            bias=False,\n            quant_config=quant_config,\n            prefix=add_prefix(\"down_proj\", prefix),\n        )\n        if hidden_act != \"silu\":\n            raise ValueError(\n                f\"Unsupported activation: {hidden_act}. \"\n                \"Only silu is supported for now.\"\n            )\n        self.act_fn = SiluAndMul()\n\n    def forward(self, x):\n        gate_up, _ = self.gate_up_proj(x)\n        x = self.act_fn(gate_up)\n        x, _ = self.down_proj(x)\n        return x\n\n\nclass Step3TextMoEMLP(nn.Module):\n    # Native\n    def __init__(\n        self,\n        layer_id: int,\n        config: Step3TextConfig,","sourceCodeStart":78,"sourceCodeEnd":114,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/step3_vl.py#L78-L114","documentation":"Step3-VL MLP, like Solar's, hardcodes SiluAndMul and rejects any hidden_act other than 'silu' at init because only the silu fused kernel is implemented.","triggerScenarios":"Loading a Step3/Step3-VL checkpoint whose config.json hidden_act != 'silu'.","commonSituations":"Fine-tuned Step3 variants or conversion tooling rewriting activation names (e.g. 'swish' vs 'silu' naming differences); partial checkpoint uploads with edited configs.","solutions":["Set hidden_act to \"silu\" in config.json (silu == swish for these models)","Update sglang if a newer release maps/activations more broadly","Verify against the original model repo's config"],"exampleFix":"// config.json\n// before: \"hidden_act\": \"gelu_new\"\n// after:  \"hidden_act\": \"silu\"","handlingStrategy":"validation","validationCode":"assert config.hidden_act in (\"silu\", \"swish\"), config.hidden_act","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate activation before launch; diff config.json against upstream"],"tags":["step3-vl","activation","config"],"backgroundTag":"unsupported-activation-function","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}