sgl-project/sglang · error · ValueError

Unsupported activation: {hidden_act}. Only silu is supported

Error message

Unsupported activation: {hidden_act}. Only silu is supported for now.

What it means

The MiMo-v2 MLP verifies hidden_act is 'silu' because it hardcodes SiluAndMul. Any other activation string in the config is rejected at module construction time.

Source

Thrown at python/sglang/srt/models/mimo_v2.py:320

            [intermediate_size] * 2,
            bias=False,
            quant_config=quant_config,
            prefix=add_prefix("gate_up_proj", prefix),
            tp_rank=tp_rank,
            tp_size=tp_size,
        )
        self.down_proj = RowParallelLinear(
            intermediate_size,
            hidden_size,
            bias=False,
            quant_config=quant_config,
            reduce_results=reduce_results,
            prefix=add_prefix("down_proj", prefix),
            tp_rank=tp_rank,
            tp_size=tp_size,
        )
        if hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {hidden_act}. "
                "Only silu is supported for now."
            )
        self.act_fn = SiluAndMul()

    def forward(
        self,
        x,
        forward_batch: ForwardBatch = None,
    ):
        if (self.tp_size == 1) and x.shape[0] == 0:
            return x

        gate_up, _ = self.gate_up_proj(x)
        x = self.act_fn(gate_up)
        x, _ = self.down_proj(x)
        return x

View on GitHub (pinned to 0132848349)

Solutions

1. Set "hidden_act": "silu" in the model's config.json
2. If the model genuinely uses another activation, add support in the MLP class instead of SiluAndMul
3. Use the correct model class matching the checkpoint's architecture

Example fix

// before
"hidden_act": "gelu_new"
// after
"hidden_act": "silu"

if config.hidden_act != 'silu':
    raise SystemExit('this model class requires hidden_act=silu')

def is_silu(cfg) -> bool:
    return getattr(cfg, 'hidden_act', None) == 'silu'

When it happens

Trigger: Constructing the MLP with config.hidden_act set to anything other than 'silu' (e.g. 'gelu', 'gelu_new') for a MiMo-v2 model.

Common situations: Fine-tuned or re-exported checkpoints with a modified config.json activation field; using the mimo_v2 model class for a derivative model with a different activation.

Related errors

• Unsupported activation: {config.hidden_act}. Only silu is su
• Unknown activation function: {act_fn}
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• Unsupported activation: {hidden_act}. Only xIELU is supporte

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4c996509baed7517. Report an issue: GitHub.