Qwen3 does not support f64; load weights as f32 or bf16 (CPU

Error message

Qwen3 does not support f64; load weights as f32 or bf16 (CPU) or f16/bf16 (GPU)

What it means

Thrown by qwen3::Model::new when the VarBuilder's dtype is f64. Qwen3's attention kernels target f32 on CPU and f16/bf16 on GPU; f64 is deliberately rejected at construction so no f64 tensor can ever reach those kernels.

Source

Thrown at candle-transformers/src/models/qwen3.rs:414

    }
}

#[derive(Debug, Clone)]
pub struct Model {
    embed_tokens: candle_nn::Embedding,
    layers: Vec<DecoderLayer>,
    norm: RmsNorm,
    device: Device,
    dtype: DType,
}

impl Model {
    pub fn new(cfg: &Config, vb: VarBuilder) -> Result<Self> {
        // f64 is not a target for Qwen3 (CPU flash runs in f32, GPU flash in f16/bf16).
        // Reject it here, at the single point where the model dtype is set, so no f64
        // tensor can ever reach the attention kernels and the inner paths never branch on it.
        if vb.dtype() == DType::F64 {
            candle::bail!(
                "Qwen3 does not support f64; load weights as f32 or bf16 (CPU) or f16/bf16 (GPU)"
            );
        }
        let embed_tokens =
            candle_nn::embedding(cfg.vocab_size, cfg.hidden_size, vb.pp("model.embed_tokens"))?;
        let rotary = Arc::new(Qwen3RotaryEmbedding::new(vb.dtype(), cfg, vb.device())?);
        let mut layers = Vec::with_capacity(cfg.num_hidden_layers);
        let vb_l = vb.pp("model.layers");
        for i in 0..cfg.num_hidden_layers {
            layers.push(DecoderLayer::new(cfg, rotary.clone(), vb_l.pp(i))?);
        }
        Ok(Self {
            embed_tokens,
            layers,
            norm: RmsNorm::new(cfg.hidden_size, cfg.rms_norm_eps, vb.pp("model.norm"))?,
            device: vb.device().clone(),
            dtype: vb.dtype(),
        })

View on GitHub (pinned to d5fee525bf)

Solutions

Construct the VarBuilder with DType::F32 for CPU inference
Use DType::BF16 or DType::F16 (as supported) for GPU inference
Convert the checkpoint to f32/f16/bf16 offline if the source weights are f64

Example fix

// before
let vb = unsafe { VarBuilder::from_mmaped_safetensors(&[path], DType::F64, &device)? };
// after
let vb = unsafe { VarBuilder::from_mmaped_safetensors(&[path], DType::F32, &device)? };

Defensive patterns

Strategy: type-guard

Validate before calling

if vb.dtype() == candle_core::DType::F64 {
    anyhow::bail!("refusing to build Qwen3 with f64; use F32/BF16");
}

Type guard

fn qwen3_compatible_dtype(d: candle_core::DType) -> bool {
    matches!(d, candle_core::DType::F32 | candle_core::DType::F16 | candle_core::DType::BF16)
}

Try / catch

match qwen3::Model::new(&cfg, vb) {
    Ok(m) => m,
    Err(e) if e.to_string().contains("does not support f64") => {
        anyhow::bail!("recreate VarBuilder with DType::F32")
    }
    Err(e) => return Err(e.into()),
}

Prevention

Explicitly pass DType::F32 (CPU) or F16/BF16 (GPU) when building VarBuilder
Never use the checkpoint's native dtype blindly
Add an assert on vb.dtype() before model construction in shared load code

When it happens

Trigger: Creating a VarBuilder (e.g. VarBuilder::from_gguf / safetensors) with dtype DType::F64 and passing it to qwen3::Model::new; forcing .to_dtype(DType::F64) on the weight source before model construction.

Common situations: Loading f64 safetensors checkpoints; defaulting to the source checkpoint dtype without specifying a supported target dtype; older code paths that used f64 as a generic precision-safe dtype.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/b549ff5293d24707. Report an issue: GitHub.