{"record":{"id":"74133a4f878c3b8e","repo":"sgl-project/sglang","slug":"norm-type-self-norm-type-not-implemented","errorCode":null,"errorMessage":"Norm type {self.norm_type} not implemented","messagePattern":"Norm type (.+?) not implemented","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/layernorm.py","lineNumber":597,"sourceCode":"    def __init__(\n        self,\n        hidden_size: int,\n        eps: float = 1e-6,\n        elementwise_affine: bool = False,\n        dtype: torch.dtype = torch.float32,\n        prefix: str = \"\",\n    ):\n        super().__init__()\n        self.eps = eps\n        self.dtype = dtype\n        if self.norm_type == \"rms\":\n            self.norm = RMSNorm(hidden_size, eps=eps, dtype=dtype)\n        elif self.norm_type == \"layer\":\n            self.norm = FP32LayerNorm(\n                hidden_size, elementwise_affine=elementwise_affine, eps=eps, dtype=dtype\n            )\n        else:\n            raise NotImplementedError(f\"Norm type {self.norm_type} not implemented\")\n\n    def forward_cuda(\n        self,\n        residual: torch.Tensor,\n        x: torch.Tensor,\n        gate: torch.Tensor | int,\n        shift: torch.Tensor,\n        scale: torch.Tensor,\n    ) -> tuple[torch.Tensor, torch.Tensor]:\n        if residual.numel() == 0 or x.numel() == 0:\n            return self.forward_native(residual, x, gate, shift, scale)\n\n        if x.shape[-1] % 256 != 0 or x.shape[-1] > 8192:\n            import warnings\n\n            warnings.warn(\n                \"FusedScaleResidualNormScaleShift cuda not available, using native fallback\",\n                stacklevel=2,","sourceCodeStart":579,"sourceCodeEnd":615,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/layernorm.py#L579-L615","documentation":"The residual-scale-shift norm wrapper only supports norm_type values 'rms' (RMSNorm) and 'layer' (FP32LayerNorm); anything else fails at construction with NotImplementedError.","triggerScenarios":"Passing norm_type not in {'rms','layer'} (e.g. 'ln', 'layer_norm', 'group') to the wrapper's __init__.","commonSituations":"Typo or different naming convention in a model config; porting a checkpoint whose config uses a different norm name; case sensitivity ('RMS' vs 'rms').","solutions":["Use 'rms' or 'layer' exactly","Normalize config: map common aliases ('rmsnorm'->'rms', 'layernorm'->'layer') before construction","Extend the __init__ branch if you genuinely need a new norm type (maintainer action)"],"exampleFix":"# before\nNormWrapper(hidden_size, norm_type=\"rmsnorm\")\n# after\nNormWrapper(hidden_size, norm_type=\"rms\")","handlingStrategy":"validation","validationCode":"assert norm_type in ('rms', 'layer'), f'unsupported norm_type: {norm_type}'","typeGuard":"def is_valid_norm_type(t: str) -> bool:\\n    return t in ('rms', 'layer')","tryCatchPattern":null,"preventionTips":["Normalize config aliases before construction","Whitelist norm types in config schema"],"tags":["layernorm","norm-type","config-validation","not-implemented"],"backgroundTag":"invalid-enum-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}