sgl-project/sglang · error · NotImplementedError

native MXFP8 MoE only supports gated swiglu-oai, got {activa

Error message

native MXFP8 MoE only supports gated swiglu-oai, got {activation=} {is_gated=}.

What it means

NotImplementedError from the AMD gfx95 native MXFP8 grouped-GEMM MoE: the single fused kernel hard-codes the MiniMax-M3 SwiGLU-OAI configuration (gated silu with gemm1_alpha/gemm1_limit). Any other activation or a non-gated MLP is rejected because no kernel variant exists for it.

Source

Thrown at python/sglang/kernels/ops/moe/mxfp8_moe_amd_gfx95.py:392

    no_combine: bool = False,
    inplace: bool = False,
    apply_router_weight_on_input: bool = False,
    routed_scaling_factor: Optional[float] = None,
    gemm1_alpha: Optional[float] = None,
    gemm1_limit: Optional[float] = None,
    swiglu_limit: Optional[float] = None,
    gate_up_interleaved: bool = True,
    expert_map: Optional[torch.Tensor] = None,
) -> torch.Tensor:
    """Native MXFP8 MoE entry (CDNA4 ``dot_scaled``).

    Keeps the SGLang ``fused_experts_mxfp8`` call contract but routes to the
    single grouped-GEMM kernel. Only the MiniMax-M3 SwiGLU-OAI (split,
    uninterleaved, gated silu with ``gemm1_alpha``/``gemm1_limit``) configuration
    is supported -- the unsupported cases below never occur on the M3 path.
    """
    if not (activation == "silu" and is_gated):
        raise NotImplementedError(
            f"native MXFP8 MoE only supports gated swiglu-oai, got "
            f"{activation=} {is_gated=}."
        )
    if b1 is not None or b2 is not None:
        raise NotImplementedError("native MXFP8 MoE does not support expert bias.")
    if apply_router_weight_on_input:
        raise NotImplementedError(
            "native MXFP8 MoE does not support apply_router_weight_on_input."
        )
    if gate_up_interleaved:
        raise NotImplementedError(
            "native MXFP8 MoE expects uninterleaved (split) gate/up layout."
        )

    # SwiGLU-OAI default activation alpha (gpt-oss); M3 may override via gemm1_alpha.
    alpha = 1.702 if gemm1_alpha is None else float(gemm1_alpha)
    beta = 1.0
    limit = None if gemm1_limit is None else float(gemm1_limit)

View on GitHub (pinned to 0132848349)

Solutions

1. Confirm the model actually uses gated silu (SwiGLU); if config says otherwise, this path cannot serve it
2. Route the model to the generic Triton fused-MoE path instead of the native MXFP8 AMD kernel
3. If you control the kernel, add support for the needed activation in the grouped-GEMM epilogue

Example fix

// before
fused_experts_mxfp8(..., activation="gelu", is_gated=False)
// after
# use the generic triton path for non-swiglu models
out = fused_experts_triton(..., activation="gelu", is_gated=False)

assert activation == "silu" and is_gated, "native MXFP8 AMD path requires gated silu"

When it happens

Trigger: Calling fused_experts_mxfp8 with activation != "silu" or is_gated=False, e.g. wiring a gelu-based or ungated MLP model onto this ROCm path.

Common situations: Porting a new MXFP8-quantized model to MI300-class hardware; a model config where activation="gelu" or the MLP has no gate; overriding activation flags from config without checking the AMD kernel's supported matrix.

Related errors

• MXFP8 MoE quantization requires SM100 or ROCm gfx95 (gfx942
• Unsupported activation: {ACTIVATION_TYPE}
• topk kernels only support streaming implementation: {_impl=}
• AITer backend does not have a metadata builder.
• AITER Sage backend does not have a metadata builder.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e7a57fc187b078bf. Report an issue: GitHub.