sgl-project/sglang · error · ValueError

Unsupported activation: {hidden_act}. Only silu is supported

Error message

Unsupported activation: {hidden_act}. Only silu is supported for now.

What it means

Solar's MLP hardcodes SiluAndMul and rejects any other hidden_act from config. The fused activation kernel only implements silu, so a config specifying gelu/pytorch_silu/etc. cannot be served correctly and init fails.

Source

Thrown at python/sglang/srt/models/solar.py:89

        prefix: str = "",
    ) -> None:
        super().__init__()
        self.gate_up_proj = MergedColumnParallelLinear(
            input_size=hidden_size,
            output_sizes=[intermediate_size] * 2,
            bias=bias,
            quant_config=quant_config,
            prefix=f"{prefix}.gate_up_proj",
        )
        self.down_proj = RowParallelLinear(
            input_size=intermediate_size,
            output_size=hidden_size,
            bias=bias,
            quant_config=quant_config,
            prefix=f"{prefix}.down_proj",
        )
        if hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {hidden_act}. "
                "Only silu is supported for now."
            )
        self.act_fn = SiluAndMul()

    def forward(self, x):
        gate_up, _ = self.gate_up_proj(x)
        x = self.act_fn(gate_up)
        x, _ = self.down_proj(x)
        return x


class SolarAttention(nn.Module):

    def __init__(
        self,
        config: PretrainedConfig,
        hidden_size: int,

View on GitHub (pinned to 0132848349)

Solutions

Edit the model's config.json hidden_act back to "silu" if the checkpoint was trained with silu
If the model genuinely uses another activation, use a backend that supports it (e.g. HF transformers) or add support in the MLP
Re-download the original checkpoint config from the model repo

Example fix

// config.json
// before: "hidden_act": "gelu"
// after:  "hidden_act": "silu"

Defensive patterns

Strategy: validation

Validate before calling

assert config.hidden_act in ("silu", "swish"), config.hidden_act

Prevention

Validate activation string against model support before launching

When it happens

Trigger: Loading a Solar-family checkpoint whose config.json hidden_act (or intermediate_act_fn) is anything other than 'silu'.

Common situations: Fine-tuned/merged Solar variants saved with a different activation string, or quantization/conversion tooling that rewrote config.json activation fields.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/319b851af3f931d1. Report an issue: GitHub.