{"record":{"id":"dcd9a41dd62301f7","repo":"huggingface/candle","slug":"shape-mismatch-in-rms-norm","errorCode":null,"errorMessage":"shape mismatch in rms-norm {:?} {:?}","messagePattern":"shape mismatch in rms-norm (.+?) (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/ops.rs","lineNumber":678,"sourceCode":"\npub fn rms_norm_slow(x: &Tensor, alpha: &Tensor, eps: f32) -> Result<Tensor> {\n    let x_dtype = x.dtype();\n    let internal_dtype = match x_dtype {\n        DType::F16 | DType::BF16 => DType::F32,\n        d => d,\n    };\n    let hidden_size = x.dim(D::Minus1)?;\n    let x = x.to_dtype(internal_dtype)?;\n    let norm_x = (x.sqr()?.sum_keepdim(D::Minus1)? / hidden_size as f64)?;\n    let x_normed = x.broadcast_div(&(norm_x + eps as f64)?.sqrt()?)?;\n    x_normed.to_dtype(x_dtype)?.broadcast_mul(alpha)\n}\n\npub fn rms_norm(xs: &Tensor, alpha: &Tensor, eps: f32) -> Result<Tensor> {\n    let hidden_size_xs = xs.dim(D::Minus1)?;\n    let hidden_size_alpha = alpha.dims1()?;\n    if hidden_size_xs != hidden_size_alpha {\n        candle::bail!(\n            \"shape mismatch in rms-norm {:?} {:?}\",\n            xs.shape(),\n            alpha.shape()\n        )\n    }\n    xs.apply_op2_no_bwd(alpha, &RmsNorm { eps })\n}\n\n#[derive(Debug, Clone)]\nstruct LayerNorm {\n    eps: f32,\n}\n\nimpl candle::CustomOp3 for LayerNorm {\n    fn name(&self) -> &'static str {\n        \"layer-norm\"\n    }\n","sourceCodeStart":660,"sourceCodeEnd":696,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/ops.rs#L660-L696","documentation":"`candle_nn::ops::rms_norm` requires `alpha` to be a 1-D tensor whose length equals the last dimension (hidden size) of `xs`. Before dispatching to the kernel it checks `xs.dim(D::Minus1) == alpha.dims1()` and bails with both shapes when they differ. This is a shape-contract violation, not a memory/layout problem.","triggerScenarios":"Calling `rms_norm(xs, alpha, eps)` where `alpha`'s element count differs from `xs.dim(D::Minus1)` — e.g. weight from a model with hidden_size 4096 fed with activations of 3200, or alpha passed with 2-D shape (dims1() would also error) or transposed.","commonSituations":"Mixing layers/weights from different model configs, loading the wrong checkpoint, off-by-one reshaping so the last dim doesn't match the weight, or forgetting that rms_norm normalizes along the last dimension.","solutions":["Verify the alpha weight length matches the last dimension of the input; load the correct weight tensor for this layer.","Reshape/transpose the input so its last dimension is the hidden size matching alpha.","Ensure the model config's hidden_size matches the checkpoint being loaded."],"exampleFix":"// before: alpha has 4096 elems, xs last dim is 3200\nlet out = rms_norm(&xs, &alpha_4096, 1e-6)?;\n// after: use the weight matching the hidden size\nlet alpha = alpha_vars.get((\"layers.0.input_layernorm\", 3200))?;\nlet out = rms_norm(&xs, &alpha, 1e-6)?;","handlingStrategy":"validation","validationCode":"// before calling rms_norm\nlet hidden = xs.dim(candle_core::D::Minus1)?;\nlet alpha_len = alpha.dims1()?; // also enforces 1-D alpha\nif hidden != alpha_len {\n    return Err(candle_core::Error::Msg(format!(\n        \"rms_norm: hidden size {hidden} != alpha len {alpha_len}\"\n    )));\n}\nlet out = candle_nn::ops::rms_norm(&xs, &alpha, eps)?;","typeGuard":"fn alpha_matches(xs: &candle_core::Tensor, alpha: &candle_core::Tensor) -> candle_core::Result<bool> {\n    Ok(alpha.dims().len() == 1 && alpha.dims()[0] == xs.dim(candle_core::D::Minus1)?)\n}","tryCatchPattern":"match candle_nn::ops::rms_norm(&xs, &alpha, eps) {\n    Ok(out) => out,\n    Err(e) if e.to_string().contains(\"shape mismatch in rms-norm\") => {\n        // log both shapes, then fail fast with config context\n        return Err(e);\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Bind norm weights to the model config's hidden_size at construction time so mismatches surface at load, not at runtime.","Check checkpoint weight names/dims against the config before running inference.","Remember rms_norm normalizes the last dimension; keep inputs as [..., hidden_size]."],"tags":["rmsnorm","shape-mismatch","tensor-shape"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}