sgl-project/sglang · error · ValueError

Unsupported activation: {activation_type}

Error message

Unsupported activation: {activation_type}

What it means

The activation() helper in inkling_common/moe.py only supports the recognized activation types (falling through to silu_and_mul); any other activation_type string reaches the final raise. It is hit from apply_group_norm_silu, forward, the eager reference resblock, and moe_tp_forward.

Source

Thrown at python/sglang/srt/models/inkling_common/moe.py:578

    gateup_output: torch.Tensor,
    topk_weights: torch.Tensor | None = None,
    use_interleaved: bool = True,
):
    if activation_type == "silu_and_mul":
        assert (
            gateup_output.is_contiguous()
        ), f"{gateup_output.shape=} {gateup_output.stride()=}"
        assert gateup_output.ndim == 2, f"{gateup_output.shape=}"
        out_dtype = None
        if gateup_output.numel() == 0:
            return gateup_output.new_zeros(
                *gateup_output.shape[:-1], gateup_output.shape[-1] // 2, dtype=out_dtype
            )

        return silu_and_mul(
            gateup_output, topk_weights, out_dtype, use_interleaved=use_interleaved
        )
    raise ValueError(f"Unsupported activation: {activation_type}")


def moe_tp_forward(
    hidden_states: torch.Tensor,
    topk_weights: torch.Tensor,
    topk_ids: torch.Tensor,
    w13_weight_E_2f_D: torch.Tensor,
    w2_weight_EDf: torch.Tensor,
    w13_bias_E_2f: torch.Tensor | None = None,
    w2_bias_ED: torch.Tensor | None = None,
    activation_type: str = "silu_and_mul",
    use_interleaved: bool = True,
) -> torch.Tensor:
    orig_shape: torch.Size = hidden_states.shape
    hidden_states_TD, topk_weights_TK, topk_ids_TK, top_k, num_experts = (
        make_forward_inputs_2d(hidden_states, topk_weights, topk_ids, w2_weight_EDf)
    )
    del hidden_states, topk_weights, topk_ids

View on GitHub (pinned to 0132848349)

Solutions

1. Check the activation_type string actually passed and correct it to a supported value (silu-family)
2. If a new activation is genuinely required, extend the dispatcher in activation() with a branch and kernel support
3. Verify the model config's hidden_act matches what the Inkling MoE path supports

Example fix

# before
out = activation(gateup, topk_weights, activation_type='relu')
# after
out = activation(gateup, topk_weights, activation_type='silu')

SUPPORTED = {'silu'}  # per dispatcher
assert activation_type in SUPPORTED, f'unsupported {activation_type}'

def is_supported_activation(name: str) -> bool:
    return name in {'silu'}

try:
    out = activation(...)
except ValueError as e:
    if 'Unsupported activation' in str(e): raise ConfigError(...) from e
    raise

When it happens

Trigger: Passing an activation_type value not handled by the if/elif chain (e.g. a typo like 'silu-mul', 'gelu', or an unsupported enum) into activation().

Common situations: Model config advertises a non-silu activation; custom model variant wired with a new activation name not yet added to the dispatcher.

Related errors

• The hpc_ops MoE runner backend runs a plain SiLU-and-mul; it
• Unsupported activation: {self.activation}
• Unsupported activation: {ACTIVATION_TYPE}
• native MXFP8 MoE only supports gated swiglu-oai, got {activa
• DeepSeekV4 CP supports moe_a2a_backend in {supported_a2a_bac

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/450740293ef645e2. Report an issue: GitHub.