{"record":{"id":"b549ff5293d24707","repo":"huggingface/candle","slug":"qwen3-does-not-support-f64-load-weights-as-f32-or","errorCode":null,"errorMessage":"Qwen3 does not support f64; load weights as f32 or bf16 (CPU) or f16/bf16 (GPU)","messagePattern":"Qwen3 does not support f64; load weights as f32 or bf16 \\(CPU\\) or f16/bf16 \\(GPU\\)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/qwen3.rs","lineNumber":414,"sourceCode":"    }\n}\n\n#[derive(Debug, Clone)]\npub struct Model {\n    embed_tokens: candle_nn::Embedding,\n    layers: Vec<DecoderLayer>,\n    norm: RmsNorm,\n    device: Device,\n    dtype: DType,\n}\n\nimpl Model {\n    pub fn new(cfg: &Config, vb: VarBuilder) -> Result<Self> {\n        // f64 is not a target for Qwen3 (CPU flash runs in f32, GPU flash in f16/bf16).\n        // Reject it here, at the single point where the model dtype is set, so no f64\n        // tensor can ever reach the attention kernels and the inner paths never branch on it.\n        if vb.dtype() == DType::F64 {\n            candle::bail!(\n                \"Qwen3 does not support f64; load weights as f32 or bf16 (CPU) or f16/bf16 (GPU)\"\n            );\n        }\n        let embed_tokens =\n            candle_nn::embedding(cfg.vocab_size, cfg.hidden_size, vb.pp(\"model.embed_tokens\"))?;\n        let rotary = Arc::new(Qwen3RotaryEmbedding::new(vb.dtype(), cfg, vb.device())?);\n        let mut layers = Vec::with_capacity(cfg.num_hidden_layers);\n        let vb_l = vb.pp(\"model.layers\");\n        for i in 0..cfg.num_hidden_layers {\n            layers.push(DecoderLayer::new(cfg, rotary.clone(), vb_l.pp(i))?);\n        }\n        Ok(Self {\n            embed_tokens,\n            layers,\n            norm: RmsNorm::new(cfg.hidden_size, cfg.rms_norm_eps, vb.pp(\"model.norm\"))?,\n            device: vb.device().clone(),\n            dtype: vb.dtype(),\n        })","sourceCodeStart":396,"sourceCodeEnd":432,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/qwen3.rs#L396-L432","documentation":"Thrown by qwen3::Model::new when the VarBuilder's dtype is f64. Qwen3's attention kernels target f32 on CPU and f16/bf16 on GPU; f64 is deliberately rejected at construction so no f64 tensor can ever reach those kernels.","triggerScenarios":"Creating a VarBuilder (e.g. VarBuilder::from_gguf / safetensors) with dtype DType::F64 and passing it to qwen3::Model::new; forcing .to_dtype(DType::F64) on the weight source before model construction.","commonSituations":"Loading f64 safetensors checkpoints; defaulting to the source checkpoint dtype without specifying a supported target dtype; older code paths that used f64 as a generic precision-safe dtype.","solutions":["Construct the VarBuilder with DType::F32 for CPU inference","Use DType::BF16 or DType::F16 (as supported) for GPU inference","Convert the checkpoint to f32/f16/bf16 offline if the source weights are f64"],"exampleFix":"// before\nlet vb = unsafe { VarBuilder::from_mmaped_safetensors(&[path], DType::F64, &device)? };\n// after\nlet vb = unsafe { VarBuilder::from_mmaped_safetensors(&[path], DType::F32, &device)? };","handlingStrategy":"type-guard","validationCode":"if vb.dtype() == candle_core::DType::F64 {\n    anyhow::bail!(\"refusing to build Qwen3 with f64; use F32/BF16\");\n}","typeGuard":"fn qwen3_compatible_dtype(d: candle_core::DType) -> bool {\n    matches!(d, candle_core::DType::F32 | candle_core::DType::F16 | candle_core::DType::BF16)\n}","tryCatchPattern":"match qwen3::Model::new(&cfg, vb) {\n    Ok(m) => m,\n    Err(e) if e.to_string().contains(\"does not support f64\") => {\n        anyhow::bail!(\"recreate VarBuilder with DType::F32\")\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Explicitly pass DType::F32 (CPU) or F16/BF16 (GPU) when building VarBuilder","Never use the checkpoint's native dtype blindly","Add an assert on vb.dtype() before model construction in shared load code"],"tags":["rust","candle","dtype","f64","qwen3"],"backgroundTag":"unsupported-dtype","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}