sgl-project/sglang · error · ValueError

Unsupported activation: {hidden_act}

Error message

Unsupported activation: {hidden_act}

What it means

KimiK3MLP.__init__ maps config.hidden_act (with activation_situ_beta variants) to a concrete activation module and only supports the enumerated set (which includes 'situ'). Any other string falls through to ValueError('Unsupported activation: {hidden_act}'), protecting the MLP from constructing with an unimplemented kernel.

Source

Thrown at python/sglang/srt/models/kimi_k3.py:336

        self.down_proj = RowParallelLinear(
            intermediate_size,
            hidden_size,
            bias=False,
            quant_config=quant_config,
            reduce_results=reduce_results,
            use_dp_attention_reduce=self._dense_attn_tp,
            prefix=f"{prefix}.down_proj",
            **_tp_kwargs,
        )
        if hidden_act == "silu":
            self.act_fn = SiluAndMul()
        elif hidden_act == "situ":
            self.act_fn = SituAndMul(
                beta=activation_situ_beta or 1.0,
                linear_beta=activation_situ_linear_beta,
            )
        else:
            raise ValueError(f"Unsupported activation: {hidden_act}")
        self._dp_attention = is_dp_attention_enabled()

    def forward(
        self,
        hidden_states: torch.Tensor,
        *,
        prefix_sum: Optional[torch.Tensor] = None,
        forward_batch: Optional[ForwardBatch] = None,
    ) -> torch.Tensor:
        # DP attention only when driven from the decoder layer (forward_batch
        # given); the shared-experts instance inside KimiK3MoE passes None and
        # runs on the already-gathered buffer.
        use_dp = (
            self._dp_attention and forward_batch is not None and not self._dense_attn_tp
        )
        if use_dp:
            local_hidden_states = hidden_states
            hidden_states = get_global_dp_buffer(get_tp_group())

View on GitHub (pinned to 0132848349)

Solutions

1. Check the checkpoint's config.json hidden_act value and set it to a supported activation ('situ' for Kimi K3).
2. Upgrade sglang to a release that supports the new activation name if the checkpoint uses a recently added one.
3. If you control the checkpoint, re-export with the supported activation or add the new activation branch in KimiK3MLP.__init__ upstream.

Example fix

// before (config.json)
{"hidden_act": "situ_v2"}

// after
{"hidden_act": "situ"}

supported = {"situ"}  # per KimiK3MLP enumeration
act = json.load(open(model_config_path))["hidden_act"]
assert act in supported, f"hidden_act {act} unsupported by KimiK3MLP"

When it happens

Trigger: Loading a Kimi K3 model whose config.json hidden_act is not in the supported list (e.g. 'silu' variants not handled here, 'gelu_new', a typo, or a new activation introduced in a newer checkpoint) — raised during model construction, before any forward.

Common situations: Newer Kimi K3 checkpoint revisions with a renamed/added activation; hand-edited configs; passing a non-Kimi config to the K3 architecture; sglang version older than the checkpoint format.

Related errors

• Unknown activation function: {act_fn}
• Activation function {act_fn_name!r} is not supported.
• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• AfmoeConfig must define `num_experts`.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/faccfb8425abd6bc. Report an issue: GitHub.