{"record":{"id":"2f417b6a792f8aa0","repo":"huggingface/candle","slug":"unsupported-dtype-for-rmsnorm","errorCode":null,"errorMessage":"unsupported dtype for rmsnorm {:?}","messagePattern":"unsupported dtype for rmsnorm (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/ops.rs","lineNumber":537,"sourceCode":"                }\n            } else {\n                src.par_chunks(dim_m1)\n                    .zip(dst.par_chunks_mut(dim_m1))\n                    .for_each(|(src, dst)| {\n                        let n = src.len();\n                        rms_row(src, alpha, n, eps, dst);\n                    });\n            }\n            let storage = candle::WithDType::to_cpu_storage_owned(dst);\n            Ok((storage, Shape::from_dims(dims)))\n        }\n\n        use CpuStorage as C;\n        match (s1, s2) {\n            (C::BF16(s1), C::BF16(s2)) => inner::<half::bf16>(s1, l1, s2, l2, eps),\n            (C::F16(s1), C::F16(s2)) => inner::<half::f16>(s1, l1, s2, l2, eps),\n            (C::F32(s1), C::F32(s2)) => inner::<f32>(s1, l1, s2, l2, eps),\n            _ => candle::bail!(\"unsupported dtype for rmsnorm {:?}\", s1.dtype()),\n        }\n    }\n\n    #[cfg(feature = \"cuda\")]\n    fn cuda_fwd(\n        &self,\n        s1: &candle::CudaStorage,\n        l1: &Layout,\n        s2: &candle::CudaStorage,\n        l2: &Layout,\n    ) -> Result<(candle::CudaStorage, Shape)> {\n        use candle::cuda_backend::cudarc::driver::{\n            CudaSlice, DeviceRepr, LaunchConfig, PushKernelArg,\n        };\n        use candle::cuda_backend::{kernel_name, kernels, Map2, WrapErr};\n        use candle::{CudaDevice, WithDType};\n\n        struct S {","sourceCodeStart":519,"sourceCodeEnd":555,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/ops.rs#L519-L555","documentation":"The CPU RMSNorm kernel is generic over BF16, F16, and F32 only. If the two input storages are any other dtype (F64, integers) or mismatched, the match falls through and bails reporting the dtype of the first input.","triggerScenarios":"Calling rms_norm on CPU where x and alpha dtypes are not one of (BF16,BF16), (F16,F16), (F32,F32) — e.g. F64 tensors, or F32 x with F16 alpha.","commonSituations":"Creating tensors from Rust f64 defaults on CPU; mixing quantized scales (F16) with F32 activations; forgetting to cast after .to_dtype conversions elsewhere.","solutions":["Cast both input and alpha to the same float dtype (F32 recommended) before rms_norm","Ensure alpha dtype matches x dtype exactly (x.to_dtype(alpha.dtype())?)","Avoid F64 for RMSNorm on CPU in candle; downcast to F32"],"exampleFix":"// before\nlet out = rms_norm(&x, &alpha, eps)?; // x: F64, alpha: F64\n// after\nlet out = rms_norm(&x.to_dtype(DType::F32)?, &alpha.to_dtype(DType::F32)?, eps)?;","handlingStrategy":"type-guard","validationCode":"fn ensure_rmsnorm_pair(x: &Tensor, alpha: &Tensor) -> Result<()> {\n    let ok = matches!((x.dtype(), alpha.dtype()),\n        (DType::BF16, DType::BF16) | (DType::F16, DType::F16) | (DType::F32, DType::F32));\n    if !ok { bail!(\"rms_norm needs matching F16/BF16/F32 dtypes, got {:?}/{:?}\", x.dtype(), alpha.dtype()); }\n    Ok(())\n}","typeGuard":"fn rmsnorm_dtype_ok(x: &Tensor, a: &Tensor) -> bool {\n    x.dtype() == a.dtype() && matches!(x.dtype(), DType::F32 | DType::F16 | DType::BF16)\n}","tryCatchPattern":"let (x, a) = if rmsnorm_dtype_ok(&x, &alpha) { (x, alpha) } else {\n    (x.to_dtype(DType::F32)?, alpha.to_dtype(DType::F32)?)\n};","preventionTips":["Keep x and alpha in the same dtype","Avoid F64 for CPU rms_norm; use F32","Cast both together at checkpoint load time"],"tags":["dtype","rmsnorm","cpu","candle"],"backgroundTag":"unsupported-dtype-for-op","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}