sgl-project/sglang · error · ValueError

Expected hidden_size to be {self.hidden_size}, but found: {h

Error message

Expected hidden_size to be {self.hidden_size}, but found: {hidden_size}

What it means

RMSNorm.forward_native validates that the last dimension of the (residual-added) input equals the hidden_size the norm was constructed with. A mismatch means the model wiring feeds tensors of the wrong width into this layer.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/layernorm.py:162

        return out

    def forward_native(
        self,
        x: torch.Tensor,
        residual: Optional[torch.Tensor] = None,
    ) -> Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]:
        if not x.is_contiguous():
            x = x.contiguous()
        orig_dtype = x.dtype
        x = x.to(torch.float32)
        if residual is not None:
            x = x + residual.to(torch.float32)
            residual = x.to(orig_dtype)

        hidden_size = x.shape[-1]
        if hidden_size != self.hidden_size:
            raise ValueError(
                "Expected hidden_size to be "
                f"{self.hidden_size}, but found: {hidden_size}"
            )

        if self.variance_size_override is None:
            x_var = x
        else:
            if hidden_size < self.variance_size_override:
                raise ValueError(
                    "Expected hidden_size to be at least "
                    f"{self.variance_size_override}, but found: {hidden_size}"
                )

            x_var = x[..., : self.variance_size_override]

        if x.device.type == "mps" and self.variance_size_override is None:
            weight = self.weight.to(dtype=torch.float32)
            x = F.rms_norm(

View on GitHub (pinned to 0132848349)

Solutions

1. Verify x.shape[-1] matches the hidden_size passed to RMSNorm's constructor
2. Fix the upstream projection to output hidden_size
3. Rebuild the model from the corrected config so all layers agree

Example fix

# before
norm = RMSNorm(hidden_size=1152)
out = norm(x_1024)
# after
norm = RMSNorm(hidden_size=1024)
out = norm(x_1024)

assert x.shape[-1] == norm.hidden_size, (x.shape[-1], norm.hidden_size)

def dims_match(x: torch.Tensor, norm) -> bool:\n    return x.shape[-1] == norm.hidden_size

When it happens

Trigger: Calling forward_native (or forward_cuda/cpu/hip which delegate to it) with x.shape[-1] != self.hidden_size; also triggered when residual addition broadcasts a residual of different width.

Common situations: Model config hidden_size changed but norm modules were built from stale config; a projection feeding the norm outputs the wrong dim; copy-paste layer config mixing widths.

Related errors

• Expected hidden_size to be at least {self.variance_size_over
• Expected hidden_size to be {self.hidden_size}, but found: {h
• {tensor_name}{context_clause} with shape {tensor.shape} cann
• Block sparse tensors{context} must have shapes (B, H, M) and
• Block sparse tensors{context} {dim_name} dim must be {tgt} o

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cdbac7afb3ef8367. Report an issue: GitHub.