huggingface/candle · error

Non contiguous layernorm is not implemented

Error message

Non contiguous layernorm is not implemented

What it means

The Metal layernorm kernel in candle-nn requires all three inputs (input tensor, alpha/scale, beta/bias) to have contiguous memory. The Metal backend's fused layernorm CustomOp only computes on dense row-major data and has no strided-layout support, so it bails instead of producing wrong results. Call layer_norm_slow for a fallback.

Source

Thrown at candle-nn/src/ops.rs:880

        s3: &candle::MetalStorage,
        l3: &Layout,
    ) -> Result<(candle::MetalStorage, Shape)> {
        use candle::backend::BackendStorage;
        let device = s1.device();
        let encoder = device.command_encoder()?;
        encoder.set_label("layernorm");
        let kernels = device.kernels();
        let name = match (s1.dtype(), s2.dtype(), s3.dtype()) {
            (DType::F32, DType::F32, DType::F32) => "layernorm_f32",
            (DType::F16, DType::F16, DType::F16) => "layernorm_f16",
            (DType::BF16, DType::BF16, DType::BF16) => "layernorm_bf16",
            (dt1, dt2, dt3) => {
                candle::bail!("layernorm is not implemented for {dt1:?} {dt2:?} {dt3:?}")
            }
        };

        if !(l1.is_contiguous() && l2.is_contiguous() && l3.is_contiguous()) {
            candle::bail!("Non contiguous layernorm is not implemented");
        }

        let last_dim = l1.dims()[l1.shape().rank() - 1];
        let elem_count = l1.shape().elem_count();
        let output = device
            .new_buffer_builder()
            .with_size_for(elem_count, s1.dtype())
            .with_label("layernorm")
            .build()?;
        candle_metal_kernels::call_layer_norm(
            device.metal_device(),
            &encoder,
            kernels,
            name,
            elem_count,
            last_dim,
            self.eps,
            s1.buffer(),

View on GitHub (pinned to d5fee525bf)

Solutions

  1. Call .contiguous() on the input tensor (and on alpha/beta if needed) before layer_norm
  2. Use candle_nn::ops::layer_norm_slow as a drop-in fallback that handles arbitrary layouts
  3. Reorder ops so the tensor is reshaped into a dense form (e.g. .reshape instead of narrow+cat) before normalization
  4. Check layouts with tensor.layout().is_contiguous() in debug code to find which of the three tensors is non-contiguous

Example fix

// before
let x = hidden_states.transpose(1, 2)?;
let out = layer_norm(&x, &alpha, &beta, 1e-5)?;
// after
let x = hidden_states.transpose(1, 2)?.contiguous()?;
let out = layer_norm(&x, &alpha, &beta, 1e-5)?;
Defensive patterns

Strategy: validation

Validate before calling

fn ensure_contiguous3(x: &Tensor, a: &Tensor, b: &Tensor) -> candle::Result<(Tensor, Tensor, Tensor)> {
    Ok((x.contiguous()?, a.contiguous()?, b.contiguous()?))
}
// call before: let (x, a, b) = ensure_contiguous3(&xs, &alpha, &beta)?;

Type guard

fn is_contiguous(t: &Tensor) -> bool { t.layout().is_contiguous() }

Try / catch

match layer_norm(&xs, &alpha, &beta, eps) {
    Ok(y) => y,
    Err(e) if e.to_string().contains("Non contiguous") => layer_norm_slow(&xs, &alpha, &beta, eps)?,
    Err(e) => return Err(e),
}

Prevention

When it happens

Trigger: Calling candle_nn::ops::layer_norm (or any op dispatching the LayerNorm CustomOp3) on a Metal device where the input tensor, alpha, or beta has a non-contiguous layout — e.g. after slice/narrow, transpose, permute, or a view that produces strides.

Common situations: Passing a transposed or sliced tensor from a previous op directly into layer_norm on Apple Silicon; loading weights stored as non-contiguous views; mixing broadcasting reshapes that leave the tensor strided.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/683f831c3facd0f1. Report an issue: GitHub.