{"record":{"id":"2d7f55286de699cd","repo":"huggingface/candle","slug":"rmsnorm-is-not-implemented-for-dt1-dt2","errorCode":null,"errorMessage":"rmsnorm is not implemented for {dt1:?} {dt2:?}","messagePattern":"rmsnorm is not implemented for (.+?) (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/ops.rs","lineNumber":627,"sourceCode":"\n    #[cfg(feature = \"metal\")]\n    fn metal_fwd(\n        &self,\n        s1: &candle::MetalStorage,\n        l1: &Layout,\n        s2: &candle::MetalStorage,\n        l2: &Layout,\n    ) -> Result<(candle::MetalStorage, Shape)> {\n        use candle::backend::BackendStorage;\n        let device = s1.device();\n        let encoder = device.command_encoder()?;\n        encoder.set_label(\"rmsnorm\");\n        let kernels = device.kernels();\n        let name = match (s1.dtype(), s2.dtype()) {\n            (DType::F32, DType::F32) => \"rmsnorm_f32\",\n            (DType::F16, DType::F16) => \"rmsnorm_f16\",\n            (DType::BF16, DType::BF16) => \"rmsnorm_bf16\",\n            (dt1, dt2) => candle::bail!(\"rmsnorm is not implemented for {dt1:?} {dt2:?}\"),\n        };\n\n        if !(l1.is_contiguous() && l2.is_contiguous()) {\n            candle::bail!(\"Non contiguous rmsnorm is not implemented\");\n        }\n\n        let last_dim = l1.dims()[l1.shape().rank() - 1];\n        let elem_count = l1.shape().elem_count();\n        let output = device\n            .new_buffer_builder()\n            .with_size_for(elem_count, s1.dtype())\n            .with_label(\"rmsnorm\")\n            .build()?;\n        candle_metal_kernels::call_rms_norm(\n            device.metal_device(),\n            &encoder,\n            kernels,\n            name,","sourceCodeStart":609,"sourceCodeEnd":645,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/ops.rs#L609-L645","documentation":"The Metal RMSNorm op dispatches to shaders rmsnorm_f32/f16/bf16, and both the input and alpha must have the same supported dtype. Any other combination (F64, integers, or mismatched dtypes) has no kernel and bails with this message naming both dtypes.","triggerScenarios":"Calling ops::rms_norm on a Metal tensor pair where (x.dtype, alpha.dtype) is not exactly (F32,F32), (F16,F16), or (BF16,BF16) — e.g. F64 input, or F32 x with BF16 alpha.","commonSituations":"F64 tensors on Apple GPU (unsupported); mixed-precision checkpoints where scales are F16 but activations F32; dtype cast applied to activations but not norm weights.","solutions":["Cast both x and alpha to the same supported dtype (F32) before rms_norm","Audit checkpoint loading so norm weights are converted with the model's compute dtype","Keep the whole model on one dtype per device to avoid mixed-dtype norm layers"],"exampleFix":"// before\nlet out = rms_norm(&x, &alpha, eps)?; // x: F32, alpha: BF16\n// after\nlet alpha = alpha.to_dtype(x.dtype())?;\nlet out = rms_norm(&x, &alpha, eps)?;","handlingStrategy":"type-guard","validationCode":"fn ensure_metal_rmsnorm(x: &Tensor, alpha: &Tensor) -> Result<()> {\n    if x.dtype() != alpha.dtype() || !matches!(x.dtype(), DType::F32 | DType::F16 | DType::BF16) {\n        bail!(\"Metal rmsnorm needs matching F32/F16/BF16 dtypes, got {:?}/{:?}\", x.dtype(), alpha.dtype());\n    }\n    Ok(())\n}","typeGuard":"fn metal_rmsnorm_ok(x: &Tensor, a: &Tensor) -> bool {\n    x.dtype() == a.dtype() && matches!(x.dtype(), DType::F32 | DType::F16 | DType::BF16)\n}","tryCatchPattern":"let alpha = if metal_rmsnorm_ok(&x, &alpha) { alpha } else { alpha.to_dtype(x.dtype())? };","preventionTips":["Cast norm weights to the model compute dtype at load","Avoid F64 entirely on Metal","Enforce one dtype per model/device; add dtype asserts in layer init"],"tags":["metal","rmsnorm","dtype","gpu","candle"],"backgroundTag":"unsupported-dtype-for-op","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}