{"record":{"id":"ac0f8ceef6295d76","repo":"sgl-project/sglang","slug":"expected-hidden-size-to-be-at-least-self-variance-ac0f8c","errorCode":null,"errorMessage":"Expected hidden_size to be at least {self.variance_size_override}, but found: {hidden_size}","messagePattern":"Expected hidden_size to be at least (.+?), but found: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/layernorm.py","lineNumber":814,"sourceCode":"            if post_residual_addition is not None:\n                x = x + post_residual_addition.to(torch.float32)\n            if self.fp32_residual:\n                residual = x.clone()\n            else:\n                residual = x.to(orig_dtype)\n\n        hidden_size = x.shape[-1]\n        if hidden_size != self.hidden_size:\n            raise ValueError(\n                \"Expected hidden_size to be \"\n                f\"{self.hidden_size}, but found: {hidden_size}\"\n            )\n\n        if self.variance_size_override is None:\n            x_var = x\n        else:\n            if hidden_size < self.variance_size_override:\n                raise ValueError(\n                    \"Expected hidden_size to be at least \"\n                    f\"{self.variance_size_override}, but found: {hidden_size}\"\n                )\n\n            x_var = x[..., : self.variance_size_override]\n\n        variance = x_var.pow(2).mean(dim=-1, keepdim=True)\n        x = x * torch.rsqrt(variance + self.variance_epsilon)\n\n        if self.cast_x_before_out_mul:\n            x = self.weight * x.to(orig_dtype)\n        else:\n            x = (x * self.weight).to(orig_dtype)\n\n        if residual is None:\n            return x\n        else:\n            return x, residual","sourceCodeStart":796,"sourceCodeEnd":832,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/layernorm.py#L796-L832","documentation":"When a layernorm is created with variance_size_override, forward_native computes statistics over only the first variance_size_override elements of the last dimension. It requires hidden_size >= variance_size_override; a smaller input triggers this ValueError, indicating the layer expects a wider activation than provided.","triggerScenarios":"Constructing the layer with variance_size_override = N and calling forward_native with x.shape[-1] < N — e.g. a hybrid-attention (Mamba/GDN) model where the mamba projection width or config disagrees with the override.","commonSituations":"Misparsed variance_size_override from a custom model config; models with per-layer variance overrides (GDN) where the checkpoint width was changed; tests feeding narrow tensors.","solutions":["Verify variance_size_override in the model/config matches the actual activation width, and fix the config","Pass an input whose last dimension is at least variance_size_override","If the override is wrong for this layer, construct the layer with the correct value (or None)"],"exampleFix":"# before\nlayer = LayerNorm(hidden_size=2048, variance_size_override=4096)\n\n# after\nlayer = LayerNorm(hidden_size=4096, variance_size_override=2048)","handlingStrategy":"validation","validationCode":"ov = layer.variance_size_override or layer.hidden_size\nassert x.shape[-1] >= ov, f\"need width >= {ov}, got {x.shape[-1]}\"\nout, res = layer.forward_native(x, residual)","typeGuard":"def satisfies_variance_override(x: torch.Tensor, layer) -> bool:\n    ov = getattr(layer, \"variance_size_override\", None)\n    return ov is None or x.shape[-1] >= ov","tryCatchPattern":"try:\n    out, res = layer.forward_native(x, residual)\nexcept ValueError as e:\n    if \"variance_size_override\" in str(e):\n        raise ValueError(f\"bad config: hidden_size {x.shape[-1]} < override; check model config\") from e\n    raise","preventionTips":["Assert variance_size_override <= hidden_size at model build time","Validate GDN/hybrid model configs against actual projection widths","Unit-test the invariant when constructing custom norms"],"tags":["layernorm","variance-override","shape-validation","gdn"],"backgroundTag":"tensor-dimension-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}