{"record":{"id":"9164d2fdc9926db0","repo":"sgl-project/sglang","slug":"expected-hidden-size-to-be-at-least-self-variance","errorCode":null,"errorMessage":"Expected hidden_size to be at least {self.variance_size_override}, but found: {hidden_size}","messagePattern":"Expected hidden_size to be at least (.+?), but found: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/layernorm.py","lineNumber":171,"sourceCode":"            x = x.contiguous()\n        orig_dtype = x.dtype\n        x = x.to(torch.float32)\n        if residual is not None:\n            x = x + residual.to(torch.float32)\n            residual = x.to(orig_dtype)\n\n        hidden_size = x.shape[-1]\n        if hidden_size != self.hidden_size:\n            raise ValueError(\n                \"Expected hidden_size to be \"\n                f\"{self.hidden_size}, but found: {hidden_size}\"\n            )\n\n        if self.variance_size_override is None:\n            x_var = x\n        else:\n            if hidden_size < self.variance_size_override:\n                raise ValueError(\n                    \"Expected hidden_size to be at least \"\n                    f\"{self.variance_size_override}, but found: {hidden_size}\"\n                )\n\n            x_var = x[..., : self.variance_size_override]\n\n        if x.device.type == \"mps\" and self.variance_size_override is None:\n            weight = self.weight.to(dtype=torch.float32)\n            x = F.rms_norm(\n                x,\n                (self.hidden_size,),\n                weight,\n                self.variance_epsilon,\n            ).to(orig_dtype)\n            if residual is None:\n                return x\n            return x, residual\n","sourceCodeStart":153,"sourceCodeEnd":189,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/layernorm.py#L153-L189","documentation":"When variance_size_override is set (variance computed over only the first N features), forward_native requires hidden_size >= variance_size_override so the slice x[..., :override] is valid.","triggerScenarios":"Constructing RMSNorm with variance_size_override=N and calling forward with hidden_size < N; typically a mis-specified partial-variance norm config.","commonSituations":"Configs where the variance window was sized for a larger hidden size than the layer's actual width; after hidden_size shrink (distillation/pruning) without updating the override.","solutions":["Set variance_size_override <= hidden_size (or None to use full width)","Update the override when hidden_size changes","Validate at model build time: assert variance_size_override is None or <= hidden_size"],"exampleFix":"# before\nnorm = RMSNorm(hidden_size=1024, variance_size_override=1280)\n# after\nnorm = RMSNorm(hidden_size=1024, variance_size_override=1024)  # or None","handlingStrategy":"validation","validationCode":"if norm.variance_size_override is not None:\n    assert x.shape[-1] >= norm.variance_size_override","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate variance_size_override <= hidden_size at construction","Re-validate override after any hidden_size change"],"tags":["layernorm","shape-mismatch","variance-override","model-config"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}