{"record":{"id":"319b851af3f931d1","repo":"sgl-project/sglang","slug":"unsupported-activation-hidden-act-only-silu-is-319b85","errorCode":null,"errorMessage":"Unsupported activation: {hidden_act}. Only silu is supported for now.","messagePattern":"Unsupported activation: (.+?)\\. Only silu is supported for now\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/solar.py","lineNumber":89,"sourceCode":"        prefix: str = \"\",\n    ) -> None:\n        super().__init__()\n        self.gate_up_proj = MergedColumnParallelLinear(\n            input_size=hidden_size,\n            output_sizes=[intermediate_size] * 2,\n            bias=bias,\n            quant_config=quant_config,\n            prefix=f\"{prefix}.gate_up_proj\",\n        )\n        self.down_proj = RowParallelLinear(\n            input_size=intermediate_size,\n            output_size=hidden_size,\n            bias=bias,\n            quant_config=quant_config,\n            prefix=f\"{prefix}.down_proj\",\n        )\n        if hidden_act != \"silu\":\n            raise ValueError(\n                f\"Unsupported activation: {hidden_act}. \"\n                \"Only silu is supported for now.\"\n            )\n        self.act_fn = SiluAndMul()\n\n    def forward(self, x):\n        gate_up, _ = self.gate_up_proj(x)\n        x = self.act_fn(gate_up)\n        x, _ = self.down_proj(x)\n        return x\n\n\nclass SolarAttention(nn.Module):\n\n    def __init__(\n        self,\n        config: PretrainedConfig,\n        hidden_size: int,","sourceCodeStart":71,"sourceCodeEnd":107,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/solar.py#L71-L107","documentation":"Solar's MLP hardcodes SiluAndMul and rejects any other hidden_act from config. The fused activation kernel only implements silu, so a config specifying gelu/pytorch_silu/etc. cannot be served correctly and init fails.","triggerScenarios":"Loading a Solar-family checkpoint whose config.json hidden_act (or intermediate_act_fn) is anything other than 'silu'.","commonSituations":"Fine-tuned/merged Solar variants saved with a different activation string, or quantization/conversion tooling that rewrote config.json activation fields.","solutions":["Edit the model's config.json hidden_act back to \"silu\" if the checkpoint was trained with silu","If the model genuinely uses another activation, use a backend that supports it (e.g. HF transformers) or add support in the MLP","Re-download the original checkpoint config from the model repo"],"exampleFix":"// config.json\n// before: \"hidden_act\": \"gelu\"\n// after:  \"hidden_act\": \"silu\"","handlingStrategy":"validation","validationCode":"assert config.hidden_act in (\"silu\", \"swish\"), config.hidden_act","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate activation string against model support before launching"],"tags":["solar","activation","config"],"backgroundTag":"unsupported-activation-function","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}