What causes "Non contiguous rmsnorm is not implemented" in huggingface/candle?

Call `.contiguous()` on the input tensor (and alpha if needed) before passing it to rms_norm.

Non contiguous rmsnorm is not implemented

Error message

Non contiguous rmsnorm is not implemented

What it means

The Metal (Apple GPU) rmsnorm kernel requires both the input tensor and the alpha (weight) tensor to be contiguous, i.e. their layouts must cover memory without strides/gaps. Candle's Metal implementation does not fall back to a strided or slow path, so when `l1.is_contiguous()` or `l2.is_contiguous()` is false it bails instead of running the kernel. Non-contiguity typically results from transposes, slicing, or broadcasting that leave a view with a non-trivial stride.

Source

Thrown at candle-nn/src/ops.rs:631

        s1: &candle::MetalStorage,
        l1: &Layout,
        s2: &candle::MetalStorage,
        l2: &Layout,
    ) -> Result<(candle::MetalStorage, Shape)> {
        use candle::backend::BackendStorage;
        let device = s1.device();
        let encoder = device.command_encoder()?;
        encoder.set_label("rmsnorm");
        let kernels = device.kernels();
        let name = match (s1.dtype(), s2.dtype()) {
            (DType::F32, DType::F32) => "rmsnorm_f32",
            (DType::F16, DType::F16) => "rmsnorm_f16",
            (DType::BF16, DType::BF16) => "rmsnorm_bf16",
            (dt1, dt2) => candle::bail!("rmsnorm is not implemented for {dt1:?} {dt2:?}"),
        };

        if !(l1.is_contiguous() && l2.is_contiguous()) {
            candle::bail!("Non contiguous rmsnorm is not implemented");
        }

        let last_dim = l1.dims()[l1.shape().rank() - 1];
        let elem_count = l1.shape().elem_count();
        let output = device
            .new_buffer_builder()
            .with_size_for(elem_count, s1.dtype())
            .with_label("rmsnorm")
            .build()?;
        candle_metal_kernels::call_rms_norm(
            device.metal_device(),
            &encoder,
            kernels,
            name,
            elem_count,
            last_dim,
            self.eps,
            s1.buffer(),

View on GitHub (pinned to d5fee525bf)

Solutions

Call `.contiguous()` on the input tensor (and alpha if needed) before passing it to rms_norm.
Check where the non-contiguous view comes from (transpose/slice) and avoid the operation or re-materialize the tensor there.
Use the CPU backend or `rms_norm_slow`, which handles arbitrary layouts, if contiguity cannot be ensured cheaply.

Example fix

// before
let x = hidden_states.transpose(1, 2)?;
let out = rms_norm(&x, &alpha, 1e-6)?;
// after
let x = hidden_states.transpose(1, 2)?.contiguous()?;
let out = rms_norm(&x, &alpha, 1e-6)?;

Defensive patterns

Strategy: validation

Validate before calling

// before calling rms_norm on Metal
fn ensure_contiguous(t: &candle_core::Tensor) -> candle_core::Result<candle_core::Tensor> {
    if t.layout().is_contiguous() { Ok(t.clone()) } else { t.contiguous() }
}
let xs = ensure_contiguous(&xs)?;
let alpha = ensure_contiguous(&alpha)?;
let out = candle_nn::ops::rms_norm(&xs, &alpha, 1e-6)?;

Type guard

fn is_contiguous_tensor(t: &candle_core::Tensor) -> bool {
    t.layout().is_contiguous()
}

Try / catch

match candle_nn::ops::rms_norm(&xs, &alpha, 1e-6) {
    Ok(out) => out,
    Err(e) if e.to_string().contains("Non contiguous rmsnorm") => {
        candle_nn::ops::rms_norm(&xs.contiguous()?, &alpha.contiguous()?, 1e-6)?
    }
    Err(e) => return Err(e),
}

Prevention

Call `.contiguous()` after any transpose/permute/slice whose result feeds a fused GPU kernel.
Load norm weights directly as 1-D contiguous tensors instead of views.
Add a debug assert on `tensor.layout().is_contiguous()` at model-layer boundaries when targeting Metal.

When it happens

Trigger: Calling `candle_nn::ops::rms_norm(xs, alpha, eps)` on a Metal device where `xs` or `alpha` is a non-contiguous view — e.g. after `.transpose()`, `.permute()`, `.narrow()`, `.slice()`, or loading a weight with unusual strides.

Common situations: Running LLM inference (e.g. Llama/Mistral RMSNorm layers) on Apple Silicon where the hidden-state tensor was transposed for attention and not made contiguous before the norm; loading safetensors weights whose layout doesn't match the expected contiguous layout.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/1d84d3ba0c23be56. Report an issue: GitHub.