{"record":{"id":"c7ab8d3b4b40d864","repo":"huggingface/candle","slug":"alpha-has-to-be-contiguous","errorCode":null,"errorMessage":"alpha has to be contiguous","messagePattern":"alpha has to be contiguous","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/ops.rs","lineNumber":478,"sourceCode":"        let eps = self.eps;\n        fn inner<\n            T: candle::WithDType\n                + num_traits::Float\n                + num_traits::AsPrimitive<f32>\n                + num_traits::FromPrimitive,\n        >(\n            src: &[T],\n            layout: &Layout,\n            alpha: &[T],\n            alpha_layout: &Layout,\n            eps: f32,\n        ) -> Result<(CpuStorage, Shape)> {\n            let src = match layout.contiguous_offsets() {\n                None => candle::bail!(\"input has to be contiguous\"),\n                Some((o1, o2)) => &src[o1..o2],\n            };\n            let alpha = match alpha_layout.contiguous_offsets() {\n                None => candle::bail!(\"alpha has to be contiguous\"),\n                Some((o1, o2)) => &alpha[o1..o2],\n            };\n            let el_count = layout.shape().elem_count();\n            let dims = layout.shape().dims();\n            let dim_m1 = dims[dims.len() - 1];\n            let n_rows = el_count / dim_m1;\n            let mut dst = vec![T::zero(); el_count];\n\n            fn rms_row<\n                T: candle::WithDType\n                    + num_traits::Float\n                    + num_traits::AsPrimitive<f32>\n                    + num_traits::FromPrimitive,\n            >(\n                src: &[T],\n                alpha: &[T],\n                n: usize,\n                eps: f32,","sourceCodeStart":460,"sourceCodeEnd":496,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/ops.rs#L460-L496","documentation":"The CPU RMSNorm kernel also requires the learnable alpha/scale parameter to be a contiguous 1-D slice; if alpha's layout is not contiguous the op bails with this message. The alpha vector is indexed directly against the last dimension.","triggerScenarios":"Calling rms_norm on CPU with an alpha tensor that is a non-contiguous view (e.g. sliced from a larger weight tensor or permuted).","commonSituations":"Sharing one big parameter buffer and slicing per-layer alphas out of it; loading weights with views instead of copies; alpha produced by another op returning a strided view.","solutions":["Make alpha contiguous: alpha.contiguous()? before the call","Store per-layer alphas as separate 1-D contiguous tensors","Verify alpha.ndim()==1 and alpha.is_contiguous() when assembling weights"],"exampleFix":"// before\nlet out = rms_norm(&x, &weights.slice(0, layer, layer+1)?, eps)?;\n// after\nlet alpha = weights.slice(0, layer, layer+1)?.contiguous()?;\nlet out = rms_norm(&x, &alpha, eps)?;","handlingStrategy":"validation","validationCode":"if alpha.layout().contiguous_offsets().is_none() {\n    alpha = alpha.contiguous()?;\n}\nlet out = rms_norm(&x, &alpha, eps)?;","typeGuard":"fn alpha_ready(a: &Tensor) -> bool { a.rank() == 1 && a.layout().contiguous_offsets().is_some() }","tryCatchPattern":"let alpha = alpha.contiguous()?; // cheap no-op if already contiguous\nlet out = rms_norm(&x, &alpha, eps)?;","preventionTips":["Store per-layer alphas as separate contiguous 1-D tensors","Avoid slicing shared fused weight buffers as norm scales","Assert alpha.is_contiguous() at module init"],"tags":["cpu","rmsnorm","contiguity","candle"],"backgroundTag":"non-contiguous-tensor","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}