huggingface/candle · error

dtype mismatch, expected {:?}, got {:?}

Error message

dtype mismatch, expected {:?}, got {:?}

What it means

Raised by MetalStorage::const_set when the tensor's dtype and the scalar's variant do not correspond, e.g. calling fill_ with an f32 value on a tensor whose dtype is F16 or U32. The backend matches on (dtype, Scalar) pairs and only proceeds when they agree exactly; anything else bails with the expected/got message. This is a caller-side dtype/scalar type error, not a missing-kernel issue.

Source

Thrown at candle-core/src/metal_backend/mod.rs:543

                    kernel_name,
                    l.dims(),
                    s,
                    l.stride(),
                    dst,
                )
                .map_err(MetalError::from)?;
            }
            Ok(())
        }
        match (self.dtype, s) {
            (DType::U8, Scalar::U8(s)) => set(self, s, l),
            (DType::U32, Scalar::U32(s)) => set(self, s, l),
            (DType::I64, Scalar::I64(s)) => set(self, s, l),
            (DType::F16, Scalar::F16(s)) => set(self, s, l),
            (DType::BF16, Scalar::BF16(s)) => set(self, s, l),
            (DType::F32, Scalar::F32(s)) => set(self, s, l),
            (DType::F64, Scalar::F64(s)) => set(self, s, l),
            _ => crate::bail!("dtype mismatch, expected {:?}, got {:?}", self.dtype, s),
        }
    }

    fn to_dtype(&self, layout: &Layout, dtype: DType) -> Result<Self> {
        let device = self.device();
        let shape = layout.shape();
        let el_count = shape.elem_count();
        let buffer = device
            .new_buffer_builder()
            .with_size_for(el_count, dtype)
            .with_label("to_dtype")
            .build()?;
        let encoder = device.command_encoder()?;
        let src = buffer_o(&self.buffer, layout, self.dtype);
        if layout.is_contiguous() {
            let kernel_name = match (self.dtype, dtype) {
                (DType::U32, DType::BF16) => "cast_u32_bf16",
                (DType::U32, DType::F16) => "cast_u32_f16",

View on GitHub (pinned to d5fee525bf)

Solutions

1. Match the scalar type to the tensor dtype: use f16::from_f32/bf16 values for F16/BF16 tensors, u32 for U32, i64 for I64.
2. Or pass the value as f64 — candle's fill_ accepts an impl trait/Into<f64> for float dtypes and converts internally when supported; check the fill_ signature you are using.
3. Inspect tensor.dtype() (printed in the error as 'expected') and adjust either the tensor dtype (to_dtype) or the scalar.
4. For unpaired dtypes like F8E4M3, avoid const_set entirely (see the unsupported const-set errors) and construct via from_vec.

Example fix

// before
let t = Tensor::zeros(shape, DType::F16, &device)?;
t.fill_(0.5f32)?; // dtype mismatch, expected F16, got F32
// after
let t = Tensor::zeros(shape, DType::F16, &device)?;
t.fill_(f16::from_f32(0.5))?;

fn check_scalar_dtype(dtype: candle_core::DType, s: &candle_core::Scalar) -> Result<(), String> {
    use candle_core::{DType, Scalar};
    let ok = matches!((dtype, s),
        (DType::U8,  Scalar::U8(_))
      | (DType::U32, Scalar::U32(_))
      | (DType::I64, Scalar::I64(_))
      | (DType::F16, Scalar::F16(_))
      | (DType::BF16, Scalar::BF16(_))
      | (DType::F32, Scalar::F32(_))
      | (DType::F64, Scalar::F64(_)));
    if ok { Ok(()) } else { Err(format!("scalar {:?} does not match tensor dtype {:?}", s, dtype)) }
}

fn scalar_matches(dtype: candle_core::DType, s: &candle_core::Scalar) -> bool {
    use candle_core::{DType, Scalar};
    matches!((dtype, s),
        (DType::U8, Scalar::U8(_)) | (DType::U32, Scalar::U32(_))
      | (DType::I64, Scalar::I64(_)) | (DType::F16, Scalar::F16(_))
      | (DType::BF16, Scalar::BF16(_)) | (DType::F32, Scalar::F32(_))
      | (DType::F64, Scalar::F64(_)))
}

match t.fill_(v) {
    Ok(t) => t,
    Err(e) if e.to_string().contains("dtype mismatch") => {
        let mut t = t.to_dtype(candle_core::DType::F32, t.device())?;
        t.fill_(v)?;
        t
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Tensor::fill_ (or const_set) with a scalar whose Rust type doesn't match the tensor's DType on Metal: e.g. fill_(0.5f32) on an F16 tensor, fill_(1i64) on a U32 tensor, or fill_ on an F8E4M3/other tensor with any scalar not listed in the match at metal_backend/mod.rs:535-543.

Common situations: Copy-pasted fill calls after a dtype change; numeric literals defaulting to f32/f64 while the tensor is F16/BF16; int vs uint confusion (I64 vs U32); generic helper code that fills tensors of mixed dtypes with the same literal.

Related errors

• Invalid where: different dtypes for values {:?} != {:?}
• all q, k, v dtypes must match.
• unsupported const-set f8e4m3
• unsupported const-set f64
• Metal contiguous to_dtype {left:?} {right:?} not implemented

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/29690a769c1dc489. Report an issue: GitHub.