{"record":{"id":"d7c60da263ac72dc","repo":"sgl-project/sglang","slug":"rmsnorm-expected-hidden-size-self-hidden-size-g","errorCode":null,"errorMessage":"RMSNorm expected hidden size {self.hidden_size}, got {original_shape[-1]}","messagePattern":"RMSNorm expected hidden size (.+?), got (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/inkling_common/norm.py","lineNumber":30,"sourceCode":"\nclass RMSNorm(nn.Module):\n    def __init__(self, hidden_size: int, eps: float = 1e-6) -> None:\n        super().__init__()\n        self.weight = nn.Parameter(torch.ones(hidden_size))\n        self.variance_epsilon = eps\n        self.hidden_size = hidden_size\n\n    def forward(self, x: torch.Tensor) -> torch.Tensor:\n        if x.numel() == 0:\n            return x\n        if not x.is_cuda or rmsnorm is None:\n            return F.rms_norm(\n                x, (self.hidden_size,), self.weight, self.variance_epsilon\n            )\n\n        original_shape = x.shape\n        if original_shape[-1] != self.hidden_size:\n            raise RuntimeError(\n                f\"RMSNorm expected hidden size {self.hidden_size}, got {original_shape[-1]}\"\n            )\n        x_2d = x.reshape(-1, self.hidden_size)\n        try:\n            y = rmsnorm(x_2d, self.weight.to(x_2d.dtype), self.variance_epsilon)\n        except (AttributeError, RuntimeError):\n            return F.rms_norm(\n                x, (self.hidden_size,), self.weight, self.variance_epsilon\n            )\n        return y.view(original_shape)\n","sourceCodeStart":12,"sourceCodeEnd":41,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/inkling_common/norm.py#L12-L41","documentation":"InklingCommonRMSNorm.forward checks that the last dimension of the input equals the layer's configured hidden_size before flattening to 2-D and calling the custom rmsnorm kernel. A mismatch would corrupt the reshape, so it fails fast with a RuntimeError.","triggerScenarios":"Feeding a tensor whose shape[-1] != self.hidden_size into the norm's forward; e.g. mismatched residual/projector width or a mis-sliced hidden state.","commonSituations":"Model code changed hidden size (or TP sharding misconfigured) so upstream projections emit a different width than the norm expects; feeding vision-width tensors into a text-model norm.","solutions":["Print/inspect x.shape and self.hidden_size at the call site to find the producing layer","Fix the upstream projection (or config hidden_size) so the last dim matches","Check tensor-parallel shard widths that feed this norm"],"exampleFix":"# before\ny = norm(x)  # x: [B, 4096], norm.hidden_size == 5120\n# after\nx = proj_to_hidden(x)  # [B, 5120]\ny = norm(x)","handlingStrategy":"validation","validationCode":"assert x.shape[-1] == norm.hidden_size, (x.shape, norm.hidden_size)","typeGuard":null,"tryCatchPattern":"try:\n    y = norm(x)\nexcept RuntimeError as e:\n    if 'RMSNorm expected hidden size' in str(e):\n        # fix upstream width\n        raise","preventionTips":["Smoke-test one forward with a dummy tensor of config hidden_size after model build"],"tags":["rmsnorm","shape-mismatch","model-architecture"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}