{"record":{"id":"1d84d3ba0c23be56","repo":"huggingface/candle","slug":"non-contiguous-rmsnorm-is-not-implemented","errorCode":null,"errorMessage":"Non contiguous rmsnorm is not implemented","messagePattern":"Non contiguous rmsnorm is not implemented","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/ops.rs","lineNumber":631,"sourceCode":"        s1: &candle::MetalStorage,\n        l1: &Layout,\n        s2: &candle::MetalStorage,\n        l2: &Layout,\n    ) -> Result<(candle::MetalStorage, Shape)> {\n        use candle::backend::BackendStorage;\n        let device = s1.device();\n        let encoder = device.command_encoder()?;\n        encoder.set_label(\"rmsnorm\");\n        let kernels = device.kernels();\n        let name = match (s1.dtype(), s2.dtype()) {\n            (DType::F32, DType::F32) => \"rmsnorm_f32\",\n            (DType::F16, DType::F16) => \"rmsnorm_f16\",\n            (DType::BF16, DType::BF16) => \"rmsnorm_bf16\",\n            (dt1, dt2) => candle::bail!(\"rmsnorm is not implemented for {dt1:?} {dt2:?}\"),\n        };\n\n        if !(l1.is_contiguous() && l2.is_contiguous()) {\n            candle::bail!(\"Non contiguous rmsnorm is not implemented\");\n        }\n\n        let last_dim = l1.dims()[l1.shape().rank() - 1];\n        let elem_count = l1.shape().elem_count();\n        let output = device\n            .new_buffer_builder()\n            .with_size_for(elem_count, s1.dtype())\n            .with_label(\"rmsnorm\")\n            .build()?;\n        candle_metal_kernels::call_rms_norm(\n            device.metal_device(),\n            &encoder,\n            kernels,\n            name,\n            elem_count,\n            last_dim,\n            self.eps,\n            s1.buffer(),","sourceCodeStart":613,"sourceCodeEnd":649,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/ops.rs#L613-L649","documentation":"The Metal (Apple GPU) rmsnorm kernel requires both the input tensor and the alpha (weight) tensor to be contiguous, i.e. their layouts must cover memory without strides/gaps. Candle's Metal implementation does not fall back to a strided or slow path, so when `l1.is_contiguous()` or `l2.is_contiguous()` is false it bails instead of running the kernel. Non-contiguity typically results from transposes, slicing, or broadcasting that leave a view with a non-trivial stride.","triggerScenarios":"Calling `candle_nn::ops::rms_norm(xs, alpha, eps)` on a Metal device where `xs` or `alpha` is a non-contiguous view — e.g. after `.transpose()`, `.permute()`, `.narrow()`, `.slice()`, or loading a weight with unusual strides.","commonSituations":"Running LLM inference (e.g. Llama/Mistral RMSNorm layers) on Apple Silicon where the hidden-state tensor was transposed for attention and not made contiguous before the norm; loading safetensors weights whose layout doesn't match the expected contiguous layout.","solutions":["Call `.contiguous()` on the input tensor (and alpha if needed) before passing it to rms_norm.","Check where the non-contiguous view comes from (transpose/slice) and avoid the operation or re-materialize the tensor there.","Use the CPU backend or `rms_norm_slow`, which handles arbitrary layouts, if contiguity cannot be ensured cheaply."],"exampleFix":"// before\nlet x = hidden_states.transpose(1, 2)?;\nlet out = rms_norm(&x, &alpha, 1e-6)?;\n// after\nlet x = hidden_states.transpose(1, 2)?.contiguous()?;\nlet out = rms_norm(&x, &alpha, 1e-6)?;","handlingStrategy":"validation","validationCode":"// before calling rms_norm on Metal\nfn ensure_contiguous(t: &candle_core::Tensor) -> candle_core::Result<candle_core::Tensor> {\n    if t.layout().is_contiguous() { Ok(t.clone()) } else { t.contiguous() }\n}\nlet xs = ensure_contiguous(&xs)?;\nlet alpha = ensure_contiguous(&alpha)?;\nlet out = candle_nn::ops::rms_norm(&xs, &alpha, 1e-6)?;","typeGuard":"fn is_contiguous_tensor(t: &candle_core::Tensor) -> bool {\n    t.layout().is_contiguous()\n}","tryCatchPattern":"match candle_nn::ops::rms_norm(&xs, &alpha, 1e-6) {\n    Ok(out) => out,\n    Err(e) if e.to_string().contains(\"Non contiguous rmsnorm\") => {\n        candle_nn::ops::rms_norm(&xs.contiguous()?, &alpha.contiguous()?, 1e-6)?\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Call `.contiguous()` after any transpose/permute/slice whose result feeds a fused GPU kernel.","Load norm weights directly as 1-D contiguous tensors instead of views.","Add a debug assert on `tensor.layout().is_contiguous()` at model-layer boundaries when targeting Metal."],"tags":["metal","rmsnorm","contiguity","gpu"],"backgroundTag":"non-contiguous-tensor-not-supported","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}