sgl-project/sglang · critical · ValueError

Unsupported activation: {config.hidden_act}. Only silu is su

Error message

Unsupported activation: {config.hidden_act}. Only silu is supported for now.

What it means

The Bailing MoE expert MLP also enforces silu activation (separate check from the dense MLP). It reads config.hidden_act during MoE block construction.

Source

Thrown at python/sglang/srt/models/bailing_moe.py:192

        layer_id: int,
        config: PretrainedConfig,
        quant_config: Optional[QuantizationConfig] = None,
        alt_stream: Optional[torch.cuda.Stream] = None,
        prefix: str = "",
    ):
        super().__init__()
        self.layer_id = layer_id
        self.alt_stream = alt_stream
        self.tp_size = get_parallel().tp_size
        self.top_k = config.num_experts_per_tok
        self.norm_topk_prob = config.norm_topk_prob
        self.hidden_size = config.hidden_size
        self.num_shared_experts = config.num_shared_experts
        self.routed_scaling_factor = getattr(config, "routed_scaling_factor", 1.0)
        self.score_function = getattr(config, "score_function", None)

        if config.hidden_act != "silu":
            raise ValueError(
                f"Unsupported activation: {config.hidden_act}. "
                "Only silu is supported for now."
            )

        # Gate always runs at half / full precision for now.
        router_dtype = getattr(config, "router_dtype", None)
        if router_dtype is None:
            self.router_dtype = None
        elif router_dtype == "fp32":
            self.router_dtype = torch.float32
        else:
            self.router_dtype = torch.bfloat16

        # TODO global_server_args.ep_num_redundant_experts is used for eplb, not supported now
        assert get_exec().moe.ep_num_redundant_experts == 0
        # check group topk
        self.num_expert_group = getattr(config, "n_group", 0)
        self.topk_group = getattr(config, "topk_group", 0)

View on GitHub (pinned to 0132848349)

Solutions

1. Set "hidden_act": "silu" in config.json
2. Use the official checkpoint unchanged

Example fix

// before
"hidden_act": "gelu"
// after
"hidden_act": "silu"

assert config.hidden_act == "silu"

When it happens

Trigger: Loading a bailing_moe config where hidden_act != "silu"; the error surfaces from the experts module even if the dense MLP check passed because both read the same field.

Common situations: Same root cause as 5147: edited configs or nonstandard Bailing variants.

Related errors

• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• Unsupported activation. Only silu is supported for now.
• Unsupported activation: {config.hidden_act}. Only silu is su
• Unsupported activation: {ACTIVATION_TYPE}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4c811ba5093ece15. Report an issue: GitHub.