{"record":{"id":"06df6a3bd0acd4dd","repo":"sgl-project/sglang","slug":"unsupported-activation-config-hidden-act-only-06df6a","errorCode":null,"errorMessage":"Unsupported activation: {config.hidden_act}. Only silu is supported for now.","messagePattern":"Unsupported activation: (.+?)\\. Only silu is supported for now\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/kimi_linear.py","lineNumber":92,"sourceCode":"        quant_config: Optional[QuantizationConfig] = None,\n        prefix: str = \"\",\n        layer_idx: int = 0,\n        alt_stream: Optional[torch.cuda.Stream] = None,\n    ):\n        super().__init__()\n        hidden_size = config.hidden_size\n        intermediate_size = config.intermediate_size\n        moe_intermediate_size = config.moe_intermediate_size\n        num_experts = config.num_experts\n        moe_renormalize = config.moe_renormalize\n        self.tp_size = get_parallel().tp_size\n        self.routed_scaling_factor = config.routed_scaling_factor\n        self.num_shared_experts = config.num_shared_experts\n        self.layer_idx = layer_idx\n        self.alt_stream = alt_stream\n\n        if config.hidden_act != \"silu\":\n            raise ValueError(\n                f\"Unsupported activation: {config.hidden_act}. \"\n                \"Only silu is supported for now.\"\n            )\n\n        # Gate always runs at half / full precision for now.\n        self.gate = ReplicatedLinear(\n            hidden_size,\n            num_experts,\n            bias=False,\n            quant_config=None,\n            prefix=f\"{prefix}.gate\",\n        )\n\n        self.gate.e_score_correction_bias = nn.Parameter(torch.empty(num_experts))\n\n        self.experts = get_moe_impl_class(quant_config)(\n            num_experts=config.n_routed_experts,\n            top_k=config.num_experts_per_token,","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/kimi_linear.py#L74-L110","documentation":"Kimi-Linear's MLP/MoE gate uses a SiLU-gated activation and hard-codes support for config.hidden_act == \"silu\" only (kimi_linear.py:92). Any other activation string raises at layer construction because the gate/projection fused kernels assume SiLU numerics.","triggerScenarios":"Loading a Kimi-Linear finetune whose config.json sets hidden_act to something other than \"silu\" (e.g. \"gelu\", \"gelu_new\").","commonSituations":"Community finetunes that swapped activations; config merging errors; checkpoints from a related architecture with gelu MLPs.","solutions":["Use an official Kimi-Linear checkpoint (hidden_act=silu)","If weights truly use another activation, you must implement the corresponding gated-MLP path - not just delete the check","Re-download config.json if it may be corrupted"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"assert cfg.hidden_act == \"silu\", cfg.hidden_act","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Reject checkpoints with non-standard hidden_act rather than patching the check away"],"tags":["kimi-linear","activation","config-validation"],"backgroundTag":"unsupported-config-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}