{"record":{"id":"fa3a8893df93b061","repo":"sgl-project/sglang","slug":"unsupported-activation-self-activation","errorCode":null,"errorMessage":"Unsupported activation: {self.activation}","messagePattern":"Unsupported activation: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_norm_gate.py","lineNumber":376,"sourceCode":"    def __init__(\n        self,\n        hidden_size: int,\n        elementwise_affine: bool = True,\n        eps: float = 1e-5,\n        activation: str = \"swish\",\n        device: torch.device | None = None,\n        dtype: torch.dtype | None = None,\n    ) -> None:\n        factory_kwargs = {\"device\": device, \"dtype\": dtype}\n        super().__init__()\n\n        self.hidden_size = hidden_size\n        self.elementwise_affine = elementwise_affine\n        self.eps = eps\n        self.activation = activation\n\n        if self.activation not in [\"swish\", \"silu\", \"sigmoid\"]:\n            raise ValueError(f\"Unsupported activation: {self.activation}\")\n\n        if elementwise_affine:\n            self.weight = nn.Parameter(torch.empty(hidden_size, **factory_kwargs))\n        else:\n            self.register_parameter(\"weight\", None)\n        self.register_parameter(\"bias\", None)\n\n    def forward(\n        self,\n        x: torch.Tensor,\n        g: torch.Tensor,\n        residual: torch.Tensor | None = None,\n        prenorm: bool = False,\n        residual_in_fp32: bool = False,\n    ) -> torch.Tensor:\n        if _use_cpu:\n            assert (\n                self.activation == \"silu\"","sourceCodeStart":358,"sourceCodeEnd":394,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_norm_gate.py#L358-L394","documentation":"FusedRMSNormGated.__init__ only accepts the activation strings \"swish\", \"silu\", or \"sigmoid\" for computing the gate multiplier inside the fused kernel. Any other string (including case variants like \"SiLU\" or newer activations like \"gelu\") raises immediately at module construction.","triggerScenarios":"Instantiating FusedRMSNormGated(hidden_size, activation=\"gelu\") or activation=\"Swish\" or activation=torch.nn.functional.silu (a function instead of a string).","commonSituations":"Copying module configs from other norm implementations that use \"gelu\"/\"relu\" gates; upgrading fla-derived code where supported activation lists differ; passing the activation as a callable instead of its string name.","solutions":["Use one of \"swish\", \"silu\", or \"sigmoid\" (swish and silu are equivalent here)","If you need another activation, subclass and apply it to the gate outside the fused kernel"],"exampleFix":"# before\nnorm = FusedRMSNormGated(hidden_size, activation=\"gelu\")\n# after\nnorm = FusedRMSNormGated(hidden_size, activation=\"gelu\".replace(\"gelu\", \"silu\"))  # or \"sigmoid\" per model spec","handlingStrategy":"validation","validationCode":"assert activation in (\"swish\", \"silu\", \"sigmoid\"), activation","typeGuard":"def is_supported_activation(act: str) -> bool:\n    return act in {\"swish\", \"silu\", \"sigmoid\"}","tryCatchPattern":null,"preventionTips":["Pass activation names as lowercase strings from a fixed vocab","Validate config strings at model-config load time, not at kernel call time"],"tags":["fla","rms-norm","activation-validation","config"],"backgroundTag":"invalid-enum-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}