What causes "Invalid where: different dtypes for values {:?} != {:?}" in huggingface/candle?

Cast one branch to match the other with to_dtype before where_cond

Invalid where: different dtypes for values {:?} != {:?}

Error message

Invalid where: different dtypes for values {:?} != {:?}

What it means

where_cond selects elements from a true-branch and false-branch tensor based on a condition; both value tensors must have the same dtype. The Metal backend explicitly checks t.dtype() != f.dtype() and bails with this message before dispatching the kernel.

Source

Thrown at candle-core/src/metal_backend/mod.rs:866

        t: &Self,
        t_l: &Layout,
        f: &Self,
        f_l: &Layout,
    ) -> Result<Self> {
        let device = self.device.clone();
        let shape = t_l.shape();
        let dims = shape.dims();
        let el = shape.elem_count();
        let dtype = t.dtype;
        let buffer = self
            .device
            .new_buffer_builder()
            .with_size_for(el, dtype)
            .with_label("where")
            .build()?;
        let encoder = self.device.command_encoder()?;
        if t.dtype() != f.dtype() {
            crate::bail!(
                "Invalid where: different dtypes for values {:?} != {:?}",
                t.dtype(),
                f.dtype()
            );
        }
        let name = match (self.dtype, t.dtype()) {
            (DType::U8, DType::F32) => "where_u8_f32",
            (DType::U32, DType::F32) => "where_u32_f32",
            (DType::U8, DType::BF16) => "where_u8_bf16",
            (DType::U8, DType::F16) => "where_u8_f16",
            (DType::U8, DType::I64) => "where_u8_i64",
            (DType::U8, DType::U32) => "where_u8_u32",
            (DType::U8, DType::U8) => "where_u8_u8",
            (left, right) => crate::bail!("Metal where_cond {left:?} {right:?} not implemented"),
        };
        let src = buffer_o(&self.buffer, layout, self.dtype);
        let t = buffer_o(&t.buffer, t_l, t.dtype);
        let f = buffer_o(&f.buffer, f_l, f.dtype);

View on GitHub (pinned to d5fee525bf)

Solutions

Cast one branch to match the other with to_dtype before where_cond
Ensure both branches come from the same compute pipeline/dtype
Cast both branches to F32, run where_cond, cast back

Example fix

// before
let out = cond.where_cond(&a, &b.to_dtype(DType::F16)?)?; // a is F32
// after
let out = cond.where_cond(&a, &b.to_dtype(DType::F32)?)?;

Defensive patterns

Strategy: validation

Validate before calling

if cond_t.dtype() != a.dtype() || cond_t.dtype() != b.dtype() {
    anyhow::bail!("where_cond branches must share dtype");
}
// or normalize:
let b = b.to_dtype(a.dtype())?;
let out = cond_t.where_cond(&a, &b)?;

Try / catch

match cond.where_cond(&a, &b) {
    Ok(y) => y,
    Err(e) if e.to_string().contains("different dtypes for values") => {
        cond.where_cond(&a, &b.to_dtype(a.dtype())?)
    }
    Err(e) => return Err(e.into()),
}

Prevention

Cast both where_cond branches to the model dtype before selection
Derive both branches from tensors of the same pipeline/dtype
Add dtype assertions in layer wrappers around conditional logic

When it happens

Trigger: Calling Tensor::where_cond on Metal with a true tensor and false tensor of different dtypes, e.g. cond.where_cond(&f32_tensor, &f16_tensor)?.

Common situations: Mixing an fp32 bias branch with an fp16 model output; merging an integer index tensor with a float tensor; results of differently-typed branches in conditional logic.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/e203182dbc17a5e2. Report an issue: GitHub.