{"record":{"id":"02979835315690a3","repo":"huggingface/candle","slug":"beta-has-to-be-contiguous","errorCode":null,"errorMessage":"beta has to be contiguous","messagePattern":"beta has to be contiguous","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/ops.rs","lineNumber":732,"sourceCode":"        >(\n            src: &[T],\n            layout: &Layout,\n            alpha: &[T],\n            alpha_layout: &Layout,\n            beta: &[T],\n            beta_layout: &Layout,\n            eps: f32,\n        ) -> Result<(CpuStorage, Shape)> {\n            let src = match layout.contiguous_offsets() {\n                None => candle::bail!(\"input has to be contiguous\"),\n                Some((o1, o2)) => &src[o1..o2],\n            };\n            let alpha = match alpha_layout.contiguous_offsets() {\n                None => candle::bail!(\"alpha has to be contiguous\"),\n                Some((o1, o2)) => &alpha[o1..o2],\n            };\n            let beta = match beta_layout.contiguous_offsets() {\n                None => candle::bail!(\"beta has to be contiguous\"),\n                Some((o1, o2)) => &beta[o1..o2],\n            };\n            let el_count = layout.shape().elem_count();\n            let dims = layout.shape().dims();\n            let dim_m1 = dims[dims.len() - 1];\n            let mut dst = vec![T::zero(); el_count];\n            src.par_chunks(dim_m1)\n                .zip(dst.par_chunks_mut(dim_m1))\n                .for_each(|(src, dst)| {\n                    let mut sum = 0f32;\n                    let mut sum2 = 0f32;\n                    for v in src {\n                        let v = v.as_();\n                        sum += v;\n                        sum2 += v * v;\n                    }\n                    let mean = sum / dim_m1 as f32;\n                    let var = sum2 / dim_m1 as f32 - mean * mean;","sourceCodeStart":714,"sourceCodeEnd":750,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/ops.rs#L714-L750","documentation":"The `beta` (bias/shift) tensor passed to the CPU layernorm/rmsnorm-with-beta kernel has a non-contiguous layout: `beta_layout.contiguous_offsets()` returned None. Like input and alpha, beta must be a contiguous view so the kernel can index it as a flat slice per row.","triggerScenarios":"Passing a beta tensor derived from transpose/slice/narrow of another tensor into the CPU layernorm op, e.g. `bias.permute(...)` or a column slice of a fused weight matrix.","commonSituations":"Extracting norm biases from a fused QKV/norm parameter block with strided slicing; converting weights between layouts during quantization or model surgery.","solutions":["Call `.contiguous()` on the beta tensor before the op.","Store biases as their own 1-D contiguous tensors instead of views into larger buffers.","Verify weight-conversion code preserves contiguity for norm parameters."],"exampleFix":"// before\nlet beta = fused.narrow(0, bias_off, h)?;\nlet out = layer_norm(&x, &alpha, &beta, eps)?;\n// after\nlet beta = fused.narrow(0, bias_off, h)?.contiguous()?;\nlet out = layer_norm(&x, &alpha, &beta, eps)?;","handlingStrategy":"validation","validationCode":"// before calling the op\nif !beta.layout().is_contiguous() {\n    let beta = beta.contiguous()?;\n}\nlet out = layer_norm(&x, &alpha, &beta, eps)?;","typeGuard":"fn is_valid_beta(t: &candle_core::Tensor) -> bool {\n    t.dims().len() == 1 && t.layout().is_contiguous()\n}","tryCatchPattern":"match layer_norm(&x, &alpha, &beta, eps) {\n    Ok(out) => out,\n    Err(e) if e.to_string().contains(\"beta has to be contiguous\") => layer_norm(&x, &alpha, &beta.contiguous()?, eps)?,\n    Err(e) => return Err(e),\n}","preventionTips":["Store norm biases as standalone contiguous 1-D tensors.","Add a one-time contiguity check when loading weights rather than per forward pass.","Avoid permute/narrow on bias tensors during model surgery."],"tags":["cpu","contiguity","layernorm","weights"],"backgroundTag":"non-contiguous-tensor-not-supported","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}