sgl-project/sglang · error · ValueError

Unsupported activation: {self.activation}

Error message

Unsupported activation: {self.activation}

What it means

HummingRunnerCore.apply_activation only supports activation 'silu' (silu_and_mul) and 'gelu' (gelu_and_mul); any other self.activation string raises ValueError.

Source

Thrown at python/sglang/srt/layers/moe/moe_runner/humming.py:406

            out_2d = outputs.view(-1, outputs.shape[-1])
            act_and_mul_triton(
                gateup_output=in_2d,
                down_input=out_2d,
                config={},
                activation="silu",
                swiglu_limit=float(self.swiglu_limit),
            )
            return
        if self.activation == "silu":
            from sgl_kernel import silu_and_mul

            silu_and_mul(inputs, outputs)
        elif self.activation == "gelu":
            from sgl_kernel import gelu_and_mul

            gelu_and_mul(inputs, outputs)
        else:
            raise ValueError(f"Unsupported activation: {self.activation}")

    def _grouped_masked_act_quant(
        self,
        gate_up: torch.Tensor,
        expert_num_tokens: torch.Tensor,
        buffers: dict[str, torch.Tensor],
    ) -> tuple[torch.Tensor, torch.Tensor | None]:
        num_experts, max_tokens, two_i = gate_up.shape
        intermediate = two_i // 2
        w2_meta = self.layer.humming_metas["w2"]
        groups = intermediate // 128
        num_threads = intermediate // 8
        use_fused_masked_act_quant = (
            w2_meta.a_dtype == dtypes.float8e4m3
            and w2_meta.input_scale_group_size == 128
            and self.activation == "silu"
            and gate_up.dtype == torch.bfloat16
            and intermediate % 256 == 0

View on GitHub (pinned to 0132848349)

Solutions

1. Use a model whose MoE activation is silu or gelu with the humming backend
2. Switch to triton runner which supports more activations
3. Add a branch for the needed activation in apply_activation (contributor fix)

assert config.activation in ('silu', 'gelu'), (
    f'humming backend only supports silu/gelu, got {config.activation}')

def humming_supported(activation: str) -> bool:
    return activation in ('silu', 'gelu')

When it happens

Trigger: Instantiating HummingRunnerCore with a MoERunnerConfig whose activation is anything other than 'silu' or 'gelu' (e.g. 'gelu_tanh', 'swiglu', or a model-specific activation string), then running a GEMM path that calls apply_activation.

Common situations: Loading a MoE model with a non-standard hidden activation with --moe-runner-backend humming; model configs that specify gelu_tanh or relu activations.

Related errors

• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• The hpc_ops MoE runner backend does not support fused shared
• The hpc_ops MoE runner backend does not support apply_router
• The hpc_ops MoE runner backend does not support no_combine (
• The hpc_ops MoE runner backend only supports FP8-quantized M

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5de0c49a4527a006. Report an issue: GitHub.