huggingface/candle · error

layernorm is not implemented for {dt1:?} {dt2:?} {dt3:?}

Error message

layernorm is not implemented for {dt1:?} {dt2:?} {dt3:?}

What it means

The Metal layernorm kernel is only implemented for matching F32×3, F16×3 and BF16×3 dtype triples across (input, alpha, beta); any other combination bails listing the three dtypes. Mixed-precision layernorm on Metal is not auto-promoted.

Source

Thrown at candle-nn/src/ops.rs:875

        &self,
        s1: &candle::MetalStorage,
        l1: &Layout,
        s2: &candle::MetalStorage,
        l2: &Layout,
        s3: &candle::MetalStorage,
        l3: &Layout,
    ) -> Result<(candle::MetalStorage, Shape)> {
        use candle::backend::BackendStorage;
        let device = s1.device();
        let encoder = device.command_encoder()?;
        encoder.set_label("layernorm");
        let kernels = device.kernels();
        let name = match (s1.dtype(), s2.dtype(), s3.dtype()) {
            (DType::F32, DType::F32, DType::F32) => "layernorm_f32",
            (DType::F16, DType::F16, DType::F16) => "layernorm_f16",
            (DType::BF16, DType::BF16, DType::BF16) => "layernorm_bf16",
            (dt1, dt2, dt3) => {
                candle::bail!("layernorm is not implemented for {dt1:?} {dt2:?} {dt3:?}")
            }
        };

        if !(l1.is_contiguous() && l2.is_contiguous() && l3.is_contiguous()) {
            candle::bail!("Non contiguous layernorm is not implemented");
        }

        let last_dim = l1.dims()[l1.shape().rank() - 1];
        let elem_count = l1.shape().elem_count();
        let output = device
            .new_buffer_builder()
            .with_size_for(elem_count, s1.dtype())
            .with_label("layernorm")
            .build()?;
        candle_metal_kernels::call_layer_norm(
            device.metal_device(),
            &encoder,
            kernels,

View on GitHub (pinned to d5fee525bf)

Solutions

  1. Cast alpha and beta to the input's dtype with `.to_dtype(xs.dtype())` before the op.
  2. Construct model weights in the model's dtype (pass `DType` when creating tensors).
  3. Compute the norm manually with upcast ops (as `rms_norm_slow` does) if mixed precision is genuinely required.

Example fix

// before: xs F16, alpha/beta F32 on Metal
let out = layer_norm(&xs, &alpha, &beta, eps)?;
// after
let (alpha, beta) = (alpha.to_dtype(DType::F16)?, beta.to_dtype(DType::F16)?);
let out = layer_norm(&xs, &alpha, &beta, eps)?;
Defensive patterns

Strategy: validation

Validate before calling

// before calling the Metal layernorm op
let dt = xs.dtype();
let (alpha, beta) = if alpha.dtype() != dt || beta.dtype() != dt {
    (alpha.to_dtype(dt)?, beta.to_dtype(dt)?)
} else {
    (alpha.clone(), beta.clone())
};
let out = layer_norm_metal(&xs, &alpha, &beta, eps)?;

Type guard

fn dtypes_match_norm3(x: &candle_core::Tensor, a: &candle_core::Tensor, b: &candle_core::Tensor) -> bool {
    use candle_core::DType::*;
    matches!(x.dtype(), F32 | F16 | BF16)
        && a.dtype() == x.dtype()
        && b.dtype() == x.dtype()
}

Try / catch

match layer_norm_metal(&xs, &alpha, &beta, eps) {
    Ok(out) => out,
    Err(e) if e.to_string().contains("layernorm is not implemented for") => {
        let dt = xs.dtype();
        layer_norm_metal(&xs, &alpha.to_dtype(dt)?, &beta.to_dtype(dt)?, eps)?
    }
    Err(e) => return Err(e),
}

Prevention

When it happens

Trigger: Calling the Metal layernorm op where input, alpha, and beta dtypes differ — e.g. F32 norm weights against F16 activations, or F64/integer tensors — through `xs.apply_op3_no_bwd` / the layer_norm custom op on a Metal device.

Common situations: Creating norm weights with default F32 while the model runs in F16/BF16 on Apple Silicon; mixing checkpoints of different precisions; numeric-experiment code using F64.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/ae91cddb12fc9ad7. Report an issue: GitHub.