sgl-project/sglang · error · ValueError

Unsupported activation: {hidden_act}. Only silu is supported

Error message

Unsupported activation: {hidden_act}. Only silu is supported.

What it means

Laguna's MLP uses a SiLU-gated gate_up projection (MergedColumnParallelLinear) and only supports hidden_act == "silu" (laguna.py:77). The fused gate/up kernels and activation are compiled assuming SiLU, so other activation strings fail fast at layer init.

Source

Thrown at python/sglang/srt/models/laguna.py:77

logger = logging.getLogger(__name__)


class LagunaMLP(nn.Module):
    def __init__(
        self,
        hidden_size: int,
        intermediate_size: int,
        hidden_act: str,
        quant_config: Optional[QuantizationConfig] = None,
        reduce_results: bool = True,
        prefix: str = "",
        tp_rank: Optional[int] = None,
        tp_size: Optional[int] = None,
    ) -> None:
        super().__init__()
        if hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {hidden_act}. Only silu is supported."
            )
        self.gate_up_proj = MergedColumnParallelLinear(
            hidden_size,
            [intermediate_size] * 2,
            bias=False,
            quant_config=quant_config,
            prefix=add_prefix("gate_up_proj", prefix),
            tp_rank=tp_rank,
            tp_size=tp_size,
        )
        self.down_proj = RowParallelLinear(
            intermediate_size,
            hidden_size,
            bias=False,
            quant_config=quant_config,
            reduce_results=reduce_results,
            prefix=add_prefix("down_proj", prefix),

View on GitHub (pinned to 0132848349)

Solutions

1. Use a checkpoint with hidden_act=silu (the architecture's design)
2. If genuinely needed, extend the gated-MLP to support the new activation and adjust fused kernels
3. Verify config.json integrity / re-download weights

assert hidden_act == "silu", hidden_act

When it happens

Trigger: Loading a Laguna checkpoint whose config hidden_act is not "silu".

Common situations: Finetuned or converted checkpoints that changed MLP activation; config mixing between model families.

Related errors

• Unknown activation function: {act_fn}
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• Unsupported activation: {hidden_act}. Only xIELU is supporte
• Unsupported activation: {hidden_act}. Arcee model in SGLang

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/97ed2d265758a18c. Report an issue: GitHub.