sgl-project/sglang · error · RuntimeError

RMSNorm expected hidden size {self.hidden_size}, got {origin

Error message

RMSNorm expected hidden size {self.hidden_size}, got {original_shape[-1]}

What it means

InklingCommonRMSNorm.forward checks that the last dimension of the input equals the layer's configured hidden_size before flattening to 2-D and calling the custom rmsnorm kernel. A mismatch would corrupt the reshape, so it fails fast with a RuntimeError.

Source

Thrown at python/sglang/srt/models/inkling_common/norm.py:30

class RMSNorm(nn.Module):
    def __init__(self, hidden_size: int, eps: float = 1e-6) -> None:
        super().__init__()
        self.weight = nn.Parameter(torch.ones(hidden_size))
        self.variance_epsilon = eps
        self.hidden_size = hidden_size

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        if x.numel() == 0:
            return x
        if not x.is_cuda or rmsnorm is None:
            return F.rms_norm(
                x, (self.hidden_size,), self.weight, self.variance_epsilon
            )

        original_shape = x.shape
        if original_shape[-1] != self.hidden_size:
            raise RuntimeError(
                f"RMSNorm expected hidden size {self.hidden_size}, got {original_shape[-1]}"
            )
        x_2d = x.reshape(-1, self.hidden_size)
        try:
            y = rmsnorm(x_2d, self.weight.to(x_2d.dtype), self.variance_epsilon)
        except (AttributeError, RuntimeError):
            return F.rms_norm(
                x, (self.hidden_size,), self.weight, self.variance_epsilon
            )
        return y.view(original_shape)

View on GitHub (pinned to 0132848349)

Solutions

1. Print/inspect x.shape and self.hidden_size at the call site to find the producing layer
2. Fix the upstream projection (or config hidden_size) so the last dim matches
3. Check tensor-parallel shard widths that feed this norm

Example fix

# before
y = norm(x)  # x: [B, 4096], norm.hidden_size == 5120
# after
x = proj_to_hidden(x)  # [B, 5120]
y = norm(x)

assert x.shape[-1] == norm.hidden_size, (x.shape, norm.hidden_size)

try:
    y = norm(x)
except RuntimeError as e:
    if 'RMSNorm expected hidden size' in str(e):
        # fix upstream width
        raise

When it happens

Trigger: Feeding a tensor whose shape[-1] != self.hidden_size into the norm's forward; e.g. mismatched residual/projector width or a mis-sliced hidden state.

Common situations: Model code changed hidden size (or TP sharding misconfigured) so upstream projections emit a different width than the norm expects; feeding vision-width tensors into a text-model norm.

Related errors

• {tensor_name}{context_clause} with shape {tensor.shape} cann
• Block sparse tensors{context} must have shapes (B, H, M) and
• Block sparse tensors{context} {dim_name} dim must be {tgt} o
• Block sparse tensors{context} must share the same m-block di
• Block sparse tensors{context} n-block dimension must be <= {

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d7c60da263ac72dc. Report an issue: GitHub.