sgl-project/sglang · error · ValueError

Unsupported activation: {config.hidden_act}. Only silu is su

Error message

Unsupported activation: {config.hidden_act}. Only silu is supported for now.

What it means

Kimi-Linear's MLP/MoE gate uses a SiLU-gated activation and hard-codes support for config.hidden_act == "silu" only (kimi_linear.py:92). Any other activation string raises at layer construction because the gate/projection fused kernels assume SiLU numerics.

Source

Thrown at python/sglang/srt/models/kimi_linear.py:92

        quant_config: Optional[QuantizationConfig] = None,
        prefix: str = "",
        layer_idx: int = 0,
        alt_stream: Optional[torch.cuda.Stream] = None,
    ):
        super().__init__()
        hidden_size = config.hidden_size
        intermediate_size = config.intermediate_size
        moe_intermediate_size = config.moe_intermediate_size
        num_experts = config.num_experts
        moe_renormalize = config.moe_renormalize
        self.tp_size = get_parallel().tp_size
        self.routed_scaling_factor = config.routed_scaling_factor
        self.num_shared_experts = config.num_shared_experts
        self.layer_idx = layer_idx
        self.alt_stream = alt_stream

        if config.hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {config.hidden_act}. "
                "Only silu is supported for now."
            )

        # Gate always runs at half / full precision for now.
        self.gate = ReplicatedLinear(
            hidden_size,
            num_experts,
            bias=False,
            quant_config=None,
            prefix=f"{prefix}.gate",
        )

        self.gate.e_score_correction_bias = nn.Parameter(torch.empty(num_experts))

        self.experts = get_moe_impl_class(quant_config)(
            num_experts=config.n_routed_experts,
            top_k=config.num_experts_per_token,

View on GitHub (pinned to 0132848349)

Solutions

1. Use an official Kimi-Linear checkpoint (hidden_act=silu)
2. If weights truly use another activation, you must implement the corresponding gated-MLP path - not just delete the check
3. Re-download config.json if it may be corrupted

assert cfg.hidden_act == "silu", cfg.hidden_act

When it happens

Trigger: Loading a Kimi-Linear finetune whose config.json sets hidden_act to something other than "silu" (e.g. "gelu", "gelu_new").

Common situations: Community finetunes that swapped activations; config merging errors; checkpoints from a related architecture with gelu MLPs.

Related errors

• Unknown activation function: {act_fn}
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• Unsupported activation: {hidden_act}. Only xIELU is supporte
• Unsupported activation: {hidden_act}. Arcee model in SGLang

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/06df6a3bd0acd4dd. Report an issue: GitHub.