sgl-project/sglang · critical · ValueError

Unsupported activation. Only silu is supported for now.

Error message

Unsupported activation. Only silu is supported for now.

What it means

The Bailing MoE dense MLP requires hidden_act == 'silu'; anything else raises immediately after the projections are built.

Source

Thrown at python/sglang/srt/models/bailing_moe.py:123

            bias=config.use_bias,
            quant_config=quant_config,
            prefix=add_prefix("gate_up_proj", prefix),
            tp_rank=tp_rank,
            tp_size=tp_size,
        )
        self.down_proj = RowParallelLinear(
            intermediate_size,
            config.hidden_size,
            bias=config.use_bias,
            reduce_results=reduce_results,
            quant_config=quant_config,
            prefix=add_prefix("down_proj", prefix),
            tp_rank=tp_rank,
            tp_size=tp_size,
        )

        if config.hidden_act != "silu":
            raise ValueError("Unsupported activation. Only silu is supported for now.")
        self.act_fn = SiluAndMul()

    def forward(
        self,
        hidden_states: torch.Tensor,
        forward_batch: Optional[ForwardBatch] = None,
    ) -> torch.Tensor:
        if (self.tp_size == 1) and hidden_states.shape[0] == 0:
            return hidden_states

        gate_up, _ = self.gate_up_proj(hidden_states)
        hidden_states = self.act_fn(gate_up)
        hidden_states, _ = self.down_proj(hidden_states)
        return hidden_states


class BailingMoEGate(nn.Module):
    def __init__(

View on GitHub (pinned to 0132848349)

Solutions

1. Set "hidden_act": "silu" in config.json
2. Verify you are using the official Bailing MoE checkpoint

Example fix

// before
"hidden_act": "gelu_pytorch_tanh"
// after
"hidden_act": "silu"

assert config.hidden_act == "silu"

When it happens

Trigger: Loading a bailing_moe checkpoint whose config.hidden_act differs from "silu".

Common situations: Custom variants of Bailing models with gelu; conversion scripts writing wrong activation names.

Related errors

• Unsupported activation: {config.hidden_act}. Only silu is su
• Unknown activation function: {act_fn}
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• Unsupported activation: {hidden_act}. Only xIELU is supporte

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/78061681fca3480d. Report an issue: GitHub.