huggingface/candle · error

unexpected dtype for {}, got {:?}, expected {dt:?}

Error message

unexpected dtype for {}, got {:?}, expected {dt:?}

What it means

The final input check compares the inferred candle DType (from the declared elem_type) with the actual dtype of the supplied tensor. If they differ, this error is thrown, printing the supplied tensor's dtype and the expected one. ONNX requires inputs to match the declared element type exactly.

Source

Thrown at candle-onnx/src/eval.rs:315

                    match &d.value {
                        Some(onnx::tensor_shape_proto::dimension::Value::DimValue(v)) => {
                            if *v as usize != dim {
                                bail!(
                                    "unexpected dim {idx} for {}, got {:?}, expected {:?}",
                                    input.name,
                                    shape.dim,
                                    tensor.shape()
                                )
                            }
                        }
                        // We do not check equality constraints for the DimParam dimensions for now.
                        Some(onnx::tensor_shape_proto::dimension::Value::DimParam(_)) | None => (),
                    }
                }
            }
        };
        if dt != tensor.dtype() {
            bail!(
                "unexpected dtype for {}, got {:?}, expected {dt:?}",
                input.name,
                tensor.dtype()
            )
        }
    }
    // The nodes are topologically sorted so we can just process them in order.
    for node in graph.node.iter() {
        let get = |input_name: &str| match values.get(input_name) {
            Some(value) => Ok(value),
            None => bail!("cannot find {input_name} for op '{}'", node.name),
        };
        let get_opt = |i: usize| {
            node.input
                .get(i)
                .filter(|s: &&String| !s.is_empty())
                .map(|s| get(s))
        };

View on GitHub (pinned to d5fee525bf)

Solutions

1. Convert the input tensor to the expected dtype printed in the error (e.g. to_dtype(DType::I64) in candle, .to(torch.int64) in PyTorch before export-time contract).
2. Align tokenization/embedding code so ids are produced as i64.
3. Fix preprocessing so image tensors match the exported dtype (f16 vs f32).
4. Add a pre-eval loop asserting tensor.dtype() against model input metadata for all inputs.

Example fix

// before
let ids = Tensor::from_vec(ids_u32, shape)?.to_dtype(DType::F32)?;
// after
let ids = Tensor::from_vec(ids_u32, shape)?.to_dtype(DType::I64)?; // model expects int64

fn dtypes_match(model: &onnx::ModelProto, inputs: &HashMap<String, Value>) -> Vec<String> {
    let mut bad = vec![];
    if let Some(g) = &model.graph {
        for i in &g.input {
            if let (Some(t), Some(onnx::type_proto::Value::TensorType(tt))) =
                (inputs.get(&i.name), &i.r#type.value)
            {
                if let Some(dt) = candle_onnx::DataType::try_from(tt.elem_type).ok()
                    .and_then(candle_onnx::eval::dtype)
                {
                    if dt != t.dtype() { bad.push(i.name.clone()); }
                }
            }
        }
    }
    bad
}

match simple_eval(&model, inputs) {
    Err(e) if e.to_string().starts_with("unexpected dtype for") => {
        anyhow::bail!("convert input to the expected dtype shown in the error (e.g. to_dtype(DType::I64))")
    }
    r => r?,
}

When it happens

Trigger: simple_eval with, e.g., an f32 tensor where the model declares int64 input (common with token ids), or an i64 tensor where the model declares f32, or f16 model inputs fed with f32 data.

Common situations: Token ids passed as f32/u32 instead of i64; images normalized into f32 arrays fed to a model exported with f16 inputs; numpy default float64 vs model float32.

Related errors

• missing input {}
• unexpected rank for {}, got {:?}, expected {:?}
• unsupported 'value' data-type {dt:?} for {}
• unsupported input type {type_:?}
• unexpected dim {idx} for {}, got {:?}, expected {:?}

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/050f838015c4a326. Report an issue: GitHub.