sgl-project/sglang · error · ValueError

Unsupported activation: {hidden_act}. Only silu is supported

Error message

Unsupported activation: {hidden_act}. Only silu is supported for now.

What it means

Step3-VL MLP, like Solar's, hardcodes SiluAndMul and rejects any hidden_act other than 'silu' at init because only the silu fused kernel is implemented.

Source

Thrown at python/sglang/srt/models/step3_vl.py:96

        prefix: str = "",
    ) -> None:
        super().__init__()
        self.gate_up_proj = MergedColumnParallelLinear(
            hidden_size,
            [intermediate_size] * 2,
            bias=False,
            quant_config=quant_config,
            prefix=add_prefix("gate_up_proj", prefix),
        )
        self.down_proj = RowParallelLinear(
            intermediate_size,
            hidden_size,
            bias=False,
            quant_config=quant_config,
            prefix=add_prefix("down_proj", prefix),
        )
        if hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {hidden_act}. "
                "Only silu is supported for now."
            )
        self.act_fn = SiluAndMul()

    def forward(self, x):
        gate_up, _ = self.gate_up_proj(x)
        x = self.act_fn(gate_up)
        x, _ = self.down_proj(x)
        return x


class Step3TextMoEMLP(nn.Module):
    # Native
    def __init__(
        self,
        layer_id: int,
        config: Step3TextConfig,

View on GitHub (pinned to 0132848349)

Solutions

1. Set hidden_act to "silu" in config.json (silu == swish for these models)
2. Update sglang if a newer release maps/activations more broadly
3. Verify against the original model repo's config

Example fix

// config.json
// before: "hidden_act": "gelu_new"
// after:  "hidden_act": "silu"

assert config.hidden_act in ("silu", "swish"), config.hidden_act

When it happens

Trigger: Loading a Step3/Step3-VL checkpoint whose config.json hidden_act != 'silu'.

Common situations: Fine-tuned Step3 variants or conversion tooling rewriting activation names (e.g. 'swish' vs 'silu' naming differences); partial checkpoint uploads with edited configs.

Related errors

• Unsupported activation type: {act_type}
• Activation function {act_fn_name!r} is not supported.
• intermediate_size must be specified for scaled activation fu
• Unsupported activation: {hidden_act}. Only silu is supported
• This browser cannot encode H.264 MP4

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/876c1fd97ca7f7d9. Report an issue: GitHub.