sgl-project/sglang · critical · ValueError

Unsupported activation: {hidden_act}. Only silu is supported

Error message

Unsupported activation: {hidden_act}. Only silu is supported for now.

What it means

The Baichuan MLP only implements SwiGLU; config.hidden_act must be exactly 'silu'. Any other value fails at initialization.

Source

Thrown at python/sglang/srt/models/baichuan.py:108

        prefix: str = "",
    ):
        super().__init__()
        self.gate_up_proj = MergedColumnParallelLinear(
            hidden_size,
            [intermediate_size] * 2,
            bias=False,
            quant_config=quant_config,
            prefix=add_prefix("gate_up_proj", prefix),
        )
        self.down_proj = RowParallelLinear(
            intermediate_size,
            hidden_size,
            bias=False,
            quant_config=quant_config,
            prefix=add_prefix("down_proj", prefix),
        )
        if hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {hidden_act}. "
                "Only silu is supported for now."
            )
        self.act_fn = SiluAndMul()

    def forward(self, x):
        gate_up, _ = self.gate_up_proj(x)
        x = self.act_fn(gate_up)
        x, _ = self.down_proj(x)
        return x


class BaiChuanAttention(nn.Module):
    """Multi-headed attention from 'Attention Is All You Need' paper"""

    def __init__(
        self,
        hidden_size: int,

View on GitHub (pinned to 0132848349)

Solutions

1. Set "hidden_act": "silu" in config.json
2. Use an unmodified Baichuan checkpoint

Example fix

// before
"hidden_act": "gelu"
// after
"hidden_act": "silu"

assert config.hidden_act == "silu"

When it happens

Trigger: Loading a Baichuan model whose config.json hidden_act is not "silu".

Common situations: Modified configs from fine-tuning pipelines; activation renamed during checkpoint conversion.

Related errors

• Unknown activation function: {act_fn}
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• Unsupported activation: {hidden_act}. Only xIELU is supporte
• Unsupported activation: {hidden_act}. Arcee model in SGLang

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f7bf0ac6854b6aed. Report an issue: GitHub.