{"record":{"id":"6f64ee429a0ef00f","repo":"sgl-project/sglang","slug":"expected-hidden-size-to-be-self-hidden-size-but-6f64ee","errorCode":null,"errorMessage":"Expected hidden_size to be {self.hidden_size}, but found: {hidden_size}","messagePattern":"Expected hidden_size to be (.+?), but found: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/layernorm.py","lineNumber":805,"sourceCode":"        post_residual_addition: Optional[torch.Tensor] = None,\n        quant_linear: Optional[nn.Module] = None,\n    ) -> Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]:\n        if not x.is_contiguous():\n            x = x.contiguous()\n        orig_dtype = self.override_orig_dtype or x.dtype\n        x = x.to(torch.float32)\n        if residual is not None:\n            x = x + residual.to(torch.float32)\n            if post_residual_addition is not None:\n                x = x + post_residual_addition.to(torch.float32)\n            if self.fp32_residual:\n                residual = x.clone()\n            else:\n                residual = x.to(orig_dtype)\n\n        hidden_size = x.shape[-1]\n        if hidden_size != self.hidden_size:\n            raise ValueError(\n                \"Expected hidden_size to be \"\n                f\"{self.hidden_size}, but found: {hidden_size}\"\n            )\n\n        if self.variance_size_override is None:\n            x_var = x\n        else:\n            if hidden_size < self.variance_size_override:\n                raise ValueError(\n                    \"Expected hidden_size to be at least \"\n                    f\"{self.variance_size_override}, but found: {hidden_size}\"\n                )\n\n            x_var = x[..., : self.variance_size_override]\n\n        variance = x_var.pow(2).mean(dim=-1, keepdim=True)\n        x = x * torch.rsqrt(variance + self.variance_epsilon)\n","sourceCodeStart":787,"sourceCodeEnd":823,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/layernorm.py#L787-L823","documentation":"forward_native on a layernorm module (with variance_size_override, e.g. GDN/partial-variance norms) validates that the last dimension of the input equals the hidden_size the layer was constructed with. A mismatch means the module was built for a different width than the activations being fed to it — typically a config/feature-mismatch bug rather than a runtime data issue.","triggerScenarios":"Constructing the layernorm with one hidden_size (from model config) and calling forward_native on an x whose x.shape[-1] differs — e.g. wrong config, quantized projection changing the feature width, or manually calling forward with a test tensor of the wrong size.","commonSituations":"Model config mismatches after editing hidden_size; unit tests constructing random inputs without matching hidden_size; variants (e.g. vision towers vs text) sharing a module with different widths.","solutions":["Ensure x.shape[-1] matches the hidden_size passed at module construction (check model config)","In tests, build inputs as torch.randn(..., module.hidden_size) instead of hardcoded sizes","Audit recent changes to hidden_size/quant config that feed this layer"],"exampleFix":"# before\nx = torch.randn(B, T, 8192, device=\"cuda\")\nout, _ = layer.forward_native(x, residual)\n\n# after\nx = torch.randn(B, T, layer.hidden_size, device=\"cuda\", dtype=layer.norm_weight.dtype)\nout, _ = layer.forward_native(x, residual)","handlingStrategy":"validation","validationCode":"assert x.shape[-1] == layer.hidden_size, (\n    f\"input width {x.shape[-1]} != layer hidden_size {layer.hidden_size}\")\nout, res = layer.forward_native(x, residual)","typeGuard":"def matches_hidden_size(x: torch.Tensor, layer) -> bool:\n    return x.ndim >= 1 and x.shape[-1] == layer.hidden_size","tryCatchPattern":"try:\n    out, res = layer.forward_native(x, residual)\nexcept ValueError as e:\n    if \"Expected hidden_size\" in str(e):\n        raise RuntimeError(f\"width mismatch feeding {type(layer).__name__}: {e}\") from e\n    raise","preventionTips":["Derive test input sizes from module attributes, not literals","Validate x.shape[-1] at module entry in debug builds","Keep model config single-sourced to avoid hidden_size drift"],"tags":["layernorm","shape-mismatch","hidden-size","validation"],"backgroundTag":"tensor-dimension-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}