{"record":{"id":"2dcaa39e06b03826","repo":"sgl-project/sglang","slug":"unsupported-activation-hidden-act-only-silu-is-2dcaa3","errorCode":null,"errorMessage":"Unsupported activation: {hidden_act}. Only silu is supported for now.","messagePattern":"Unsupported activation: (.+?)\\. Only silu is supported for now\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/exaone.py","lineNumber":71,"sourceCode":"        prefix: str = \"\",\n    ) -> None:\n        super().__init__()\n        self.gate_up_proj = MergedColumnParallelLinear(\n            hidden_size,\n            [intermediate_size] * 2,\n            bias=False,\n            quant_config=quant_config,\n            prefix=add_prefix(\"gate_up_proj\", prefix),\n        )\n        self.c_proj = RowParallelLinear(\n            intermediate_size,\n            hidden_size,\n            bias=False,\n            quant_config=quant_config,\n            prefix=add_prefix(\"c_proj\", prefix),\n        )\n        if hidden_act != \"silu\":\n            raise ValueError(\n                f\"Unsupported activation: {hidden_act}. \"\n                \"Only silu is supported for now.\"\n            )\n        self.act_fn = SiluAndMul()\n\n    def forward(self, x):\n        gate_up, _ = self.gate_up_proj(x)\n        x = self.act_fn(gate_up)\n        x, _ = self.c_proj(x)\n        return x\n\n\nclass ExaoneAttention(nn.Module):\n    def __init__(\n        self,\n        config,\n        hidden_size: int,\n        num_heads: int,","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/exaone.py#L53-L89","documentation":"The EXAONE dense model's MLP module hard-codes SiLU-and-gated MLP semantics via SiluAndMul, so any other hidden activation in the HF config is unsupported. The __init__ of the MLP class validates config.hidden_activation and raises immediately if it is not 'silu'. This is a model-architecture compatibility guard, not a runtime fault.","triggerScenarios":"Loading an EXAONE checkpoint whose config.json has hidden_activation other than 'silu' (e.g. 'gelu', 'gelu_pytorch_tanh', 'silu_pytorch1') so ExaoneMLP.__init__ fails during model construction.","commonSituations":"Using a new EXAONE variant (e.g. a non-gated or GeLU-based release) with the current sglang implementation; a hand-edited or converted config.json where the activation string was changed; a community fine-tune with a modified architecture.","solutions":["Check config.json of the checkpoint and confirm hidden_activation is exactly \"silu\" (also verify hidden_act if present).","If the model genuinely uses a different activation, extend the MLP to use the matching act (e.g. GeluAndMul) instead of SiluAndMul and relax the check — see how gemma2.py gates GeluAndMul.","Use an official EXAONE checkpoint matching the supported architecture rather than a modified one."],"exampleFix":"// before (config.json)\n\"hidden_activation\": \"gelu_pytorch_tanh\"\n// after\n\"hidden_activation\": \"silu\"","handlingStrategy":"validation","validationCode":"import json\ncfg = json.load(open(\"config.json\"))\nassert cfg.get(\"hidden_activation\", \"silu\") == \"silu\", f\"unsupported activation: {cfg.get('hidden_activation')}\"","typeGuard":"def is_silu_config(cfg: dict) -> bool:\n    return cfg.get(\"hidden_activation\", \"silu\") == \"silu\"","tryCatchPattern":null,"preventionTips":["Validate checkpoint config.json activation fields before launching sglang.","Use official EXAONE checkpoints; treat edited configs with suspicion."],"tags":["exaone","activation","model-config","unsupported-operation"],"backgroundTag":"unsupported-activation-function","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}