What causes "dtype mismatch" in huggingface/candle?

Cast all tensors to a common dtype before cat: tensors.iter().map(|t| t.to_dtype(DType::F32)).collect().

dtype mismatch

Error message

dtype mismatch

What it means

When concatenating CPU storages (cat op), candle groups input storages by dtype. All inputs must have the same storage variant as the first tensor (here U8); any tensor whose storage is not the expected variant triggers this bail. It guards against mixing dtypes in cat/concat.

Source

Thrown at candle-core/src/cpu_backend/mod.rs:1758

    } else {
        (v.exp() - T::one()) * alpha
    }
}

impl CpuStorage {
    pub fn as_slice<D: WithDType>(&self) -> Result<&[D]> {
        D::cpu_storage_as_slice(self)
    }

    pub fn concat(storages: &[CpuStorage]) -> Result<CpuStorage> {
        let storage0 = &storages[0];
        let s = match storage0 {
            Self::U8(_) => {
                let storages = storages
                    .iter()
                    .map(|s| match s {
                        Self::U8(s) => Ok(s.as_slice()),
                        _ => crate::bail!("dtype mismatch"),
                    })
                    .collect::<Result<Vec<_>>>()?
                    .concat();
                Self::U8(storages)
            }
            Self::U32(_) => {
                let storages = storages
                    .iter()
                    .map(|s| match s {
                        Self::U32(s) => Ok(s.as_slice()),
                        _ => crate::bail!("dtype mismatch"),
                    })
                    .collect::<Result<Vec<_>>>()?
                    .concat();
                Self::U32(storages)
            }
            Self::I16(_) => {
                let storages = storages

View on GitHub (pinned to d5fee525bf)

Solutions

Cast all tensors to a common dtype before cat: tensors.iter().map(|t| t.to_dtype(DType::F32)).collect().
Check the dtype of every input with t.dtype() and fix the source of the divergent tensor.
Move conversions (to_dtype) into the producing code instead of at the concat site.
If dtype divergence is intentional, use a different op (e.g. stack in a common dtype or keep separate tensors).

Example fix

// before
let cat = Tensor::cat(&[&u8_tensor, &f32_tensor], 0)?;
// after
let cat = Tensor::cat(&[&u8_tensor.to_dtype(DType::F32)?, &f32_tensor], 0)?;

Defensive patterns

Strategy: validation

Validate before calling

let dt = tensors[0].dtype();
for t in &tensors { if t.dtype() != dt { return Err(anyhow::anyhow!("cat: mixed dtypes {:?} vs {:?}", dt, t.dtype())); } }

Try / catch

match result { Err(e) if e.to_string().contains("dtype mismatch") => { // unify dtypes and retry
}, other => other?, }

Prevention

Normalize every tensor to a single pipeline dtype (usually f32) before concat.
Assert homogeneous dtypes in helper functions that accept tensor lists.
Trace where u8 image tensors and f32 tensors meet; insert to_dtype at boundaries.

When it happens

Trigger: Tensor::cat(&tensors, dim) (or cat-related paths used by book_hub_1/book_hub_2) where the first tensor is u8 but at least one other tensor has a different dtype (f32, u32, i16, ...).

Common situations: Concatenating an image tensor (u8) with a normalized/converted tensor (f32); mixing indices (u32) with data tensors; a pipeline step that forgot a to_dtype conversion.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/72670474f5ea39b8. Report an issue: GitHub.