sgl-project/sglang · error · ValueError

Unsupported activation: {self.activation}

Error message

Unsupported activation: {self.activation}

What it means

FusedRMSNormGated.__init__ only accepts the activation strings "swish", "silu", or "sigmoid" for computing the gate multiplier inside the fused kernel. Any other string (including case variants like "SiLU" or newer activations like "gelu") raises immediately at module construction.

Source

Thrown at python/sglang/kernels/ops/attention/fla/fused_norm_gate.py:376

    def __init__(
        self,
        hidden_size: int,
        elementwise_affine: bool = True,
        eps: float = 1e-5,
        activation: str = "swish",
        device: torch.device | None = None,
        dtype: torch.dtype | None = None,
    ) -> None:
        factory_kwargs = {"device": device, "dtype": dtype}
        super().__init__()

        self.hidden_size = hidden_size
        self.elementwise_affine = elementwise_affine
        self.eps = eps
        self.activation = activation

        if self.activation not in ["swish", "silu", "sigmoid"]:
            raise ValueError(f"Unsupported activation: {self.activation}")

        if elementwise_affine:
            self.weight = nn.Parameter(torch.empty(hidden_size, **factory_kwargs))
        else:
            self.register_parameter("weight", None)
        self.register_parameter("bias", None)

    def forward(
        self,
        x: torch.Tensor,
        g: torch.Tensor,
        residual: torch.Tensor | None = None,
        prenorm: bool = False,
        residual_in_fp32: bool = False,
    ) -> torch.Tensor:
        if _use_cpu:
            assert (
                self.activation == "silu"

View on GitHub (pinned to 0132848349)

Solutions

1. Use one of "swish", "silu", or "sigmoid" (swish and silu are equivalent here)
2. If you need another activation, subclass and apply it to the gate outside the fused kernel

Example fix

# before
norm = FusedRMSNormGated(hidden_size, activation="gelu")
# after
norm = FusedRMSNormGated(hidden_size, activation="gelu".replace("gelu", "silu"))  # or "sigmoid" per model spec

assert activation in ("swish", "silu", "sigmoid"), activation

def is_supported_activation(act: str) -> bool:
    return act in {"swish", "silu", "sigmoid"}

When it happens

Trigger: Instantiating FusedRMSNormGated(hidden_size, activation="gelu") or activation="Swish" or activation=torch.nn.functional.silu (a function instead of a string).

Common situations: Copying module configs from other norm implementations that use "gelu"/"relu" gates; upgrading fla-derived code where supported activation lists differ; passing the activation as a callable instead of its string name.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• This layer norm doesn't support feature dim >= 64KB.
• This browser cannot encode H.264 MP4
• H.264 encoder did not return MP4 decoder config
• This browser does not support gzip stream decoding
• delta payload size mismatch: expected ${expectedSize}, got $

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fa3a8893df93b061. Report an issue: GitHub.