sgl-project/sglang · critical · ValueError

Unsupported activation: {hidden_act}. Only xIELU is supporte

Error message

Unsupported activation: {hidden_act}. Only xIELU is supported for now.

What it means

The Apertus MLP only implements the xIELU activation; the config's hidden_act must be exactly 'xielu'. Any other activation string raises at model construction.

Source

Thrown at python/sglang/srt/models/apertus.py:89

    ) -> None:
        super().__init__()
        self.up_proj = ColumnParallelLinear(
            hidden_size,
            intermediate_size,
            bias=bias,
            quant_config=quant_config,
            prefix=add_prefix("up_proj", prefix),
        )
        self.down_proj = RowParallelLinear(
            intermediate_size,
            hidden_size,
            bias=bias,
            quant_config=quant_config,
            prefix=add_prefix("down_proj", prefix),
            reduce_results=reduce_results,
        )
        if hidden_act != "xielu":
            raise ValueError(
                f"Unsupported activation: {hidden_act}. "
                "Only xIELU is supported for now."
            )
        self.act_fn = XIELU()

    def forward(
        self,
        x,
        forward_batch=None,
    ):
        # note: with xielu, there's no gate_proj
        x, _ = self.up_proj(x)
        x = self.act_fn(x)
        x, _ = self.down_proj(x)
        return x


class ApertusAttention(nn.Module):

View on GitHub (pinned to 0132848349)

Solutions

1. Set "hidden_act": "xielu" in config.json
2. Use a checkpoint that actually uses xIELU; do not point Apertus code at a silu-based variant

Example fix

// before
"hidden_act": "silu"
// after
"hidden_act": "xielu"

assert config.hidden_act == "xielu", f"Apertus requires xielu, got {config.hidden_act}"

When it happens

Trigger: Loading an Apertus checkpoint whose config.json hidden_act is anything but "xielu" (e.g. "silu", "gelu").

Common situations: Fine-tuned Apertus variants that swapped the activation; checkpoint conversion tools writing the default HF activation name.

Related errors

• Unknown activation function: {act_fn}
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• Unsupported activation: {hidden_act}. Arcee model in SGLang
• Unsupported activation: {hidden_act}. Only silu is supported

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2cd022b42919e090. Report an issue: GitHub.