sgl-project/sglang · error · ValueError

Unsupported activation: {hidden_act}. Only silu is supported

Error message

Unsupported activation: {hidden_act}. Only silu is supported for now.

What it means

The EXAONE dense model's MLP module hard-codes SiLU-and-gated MLP semantics via SiluAndMul, so any other hidden activation in the HF config is unsupported. The __init__ of the MLP class validates config.hidden_activation and raises immediately if it is not 'silu'. This is a model-architecture compatibility guard, not a runtime fault.

Source

Thrown at python/sglang/srt/models/exaone.py:71

        prefix: str = "",
    ) -> None:
        super().__init__()
        self.gate_up_proj = MergedColumnParallelLinear(
            hidden_size,
            [intermediate_size] * 2,
            bias=False,
            quant_config=quant_config,
            prefix=add_prefix("gate_up_proj", prefix),
        )
        self.c_proj = RowParallelLinear(
            intermediate_size,
            hidden_size,
            bias=False,
            quant_config=quant_config,
            prefix=add_prefix("c_proj", prefix),
        )
        if hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {hidden_act}. "
                "Only silu is supported for now."
            )
        self.act_fn = SiluAndMul()

    def forward(self, x):
        gate_up, _ = self.gate_up_proj(x)
        x = self.act_fn(gate_up)
        x, _ = self.c_proj(x)
        return x


class ExaoneAttention(nn.Module):
    def __init__(
        self,
        config,
        hidden_size: int,
        num_heads: int,

View on GitHub (pinned to 0132848349)

Solutions

1. Check config.json of the checkpoint and confirm hidden_activation is exactly "silu" (also verify hidden_act if present).
2. If the model genuinely uses a different activation, extend the MLP to use the matching act (e.g. GeluAndMul) instead of SiluAndMul and relax the check — see how gemma2.py gates GeluAndMul.
3. Use an official EXAONE checkpoint matching the supported architecture rather than a modified one.

Example fix

// before (config.json)
"hidden_activation": "gelu_pytorch_tanh"
// after
"hidden_activation": "silu"

import json
cfg = json.load(open("config.json"))
assert cfg.get("hidden_activation", "silu") == "silu", f"unsupported activation: {cfg.get('hidden_activation')}"

def is_silu_config(cfg: dict) -> bool:
    return cfg.get("hidden_activation", "silu") == "silu"

When it happens

Trigger: Loading an EXAONE checkpoint whose config.json has hidden_activation other than 'silu' (e.g. 'gelu', 'gelu_pytorch_tanh', 'silu_pytorch1') so ExaoneMLP.__init__ fails during model construction.

Common situations: Using a new EXAONE variant (e.g. a non-gated or GeLU-based release) with the current sglang implementation; a hand-edited or converted config.json where the activation string was changed; a community fine-tune with a modified architecture.

Related errors

• Unsupported activation: {ACTIVATION_TYPE}
• Activation function {act_fn_name!r} is not supported.
• FA4 does not support updating KV cache in-place.
• FA4 path does not support rotary embedding.
• FA4 path does not support non-consecutive batch indices or l

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2dcaa39e06b03826. Report an issue: GitHub.