huggingface/candle · error

unexpected rank for {}, got {:?}, expected {:?}

Error message

unexpected rank for {}, got {:?}, expected {:?}

What it means

simple_eval_ validates each supplied input tensor against the declared shape in graph.input.value_info. If the tensor's rank (number of dimensions) differs from the number of dims declared in the ONNX TensorShapeProto, this error is thrown, printing the declared dims and the actual tensor shape.

Source

Thrown at candle-onnx/src/eval.rs:289

        let tensor = match values.get(&input.name) {
            None => bail!("missing input {}", input.name),
            Some(tensor) => tensor,
        };
        let dt = match DataType::try_from(tensor_type.elem_type) {
            Ok(dt) => match dtype(dt) {
                Some(dt) => dt,
                None => {
                    bail!("unsupported 'value' data-type {dt:?} for {}", input.name)
                }
            },
            type_ => bail!("unsupported input type {type_:?}"),
        };
        match &tensor_type.shape {
            None => continue,
            Some(shape) => {
                if shape.dim.len() != tensor.rank() {
                    bail!(
                        "unexpected rank for {}, got {:?}, expected {:?}",
                        input.name,
                        shape.dim,
                        tensor.shape()
                    )
                }
                for (idx, (d, &dim)) in shape.dim.iter().zip(tensor.dims().iter()).enumerate() {
                    match &d.value {
                        Some(onnx::tensor_shape_proto::dimension::Value::DimValue(v)) => {
                            if *v as usize != dim {
                                bail!(
                                    "unexpected dim {idx} for {}, got {:?}, expected {:?}",
                                    input.name,
                                    shape.dim,
                                    tensor.shape()
                                )
                            }
                        }

View on GitHub (pinned to d5fee525bf)

Solutions

1. Compare the declared shape (printed in the error) with your tensor's shape and reshape (unsqueeze/squeeze) before eval.
2. If the model supports dynamic axes, re-export with dynamic_axes so rank checks match flexible usage.
3. Fix off-by-one on batch: wrap the tensor in an explicit batch dimension of 1 when needed.
4. Add pre-eval shape assertions in your app mirroring the model's input metadata.

Example fix

// before
let x = Tensor::new(vec![...], &dev)?; // rank 2
// after
let x = Tensor::new(vec![vec![...]], &dev)?.unsqueeze(0)?; // rank 3 as declared

fn ranks_match(model: &onnx::ModelProto, inputs: &HashMap<String, Value>) -> bool {
    model.graph.as_ref().map_or(true, |g| g.input.iter().all(|i| {
        match (inputs.get(&i.name), &i.r#type.value) {
            (Some(t), Some(onnx::type_proto::Value::TensorType(tt))) => match &tt.shape {
                Some(s) => s.dim.len() == t.rank(),
                None => true,
            },
            _ => true,
        }
    }))
}

match simple_eval(&model, inputs) {
    Err(e) if e.to_string().starts_with("unexpected rank for") => {
        anyhow::bail!("reshape inputs to match the declared rank (see error for expected dims)")
    }
    r => r?,
}

When it happens

Trigger: Calling simple_eval with an input tensor whose .rank() != shape.dim.len() for a graph input that declares a fixed-rank shape — e.g. passing a [B,S] tensor where [B,S,H] is declared, or omitting/adding a batch dimension.

Common situations: Forgetting the batch dimension (model expects [batch, seq], caller passes [seq]); squeezing/unsqueezing differences between PyTorch export and candle tensors; dynamic axes exported as fixed dims.

Related errors

• missing input {}
• unexpected dim {idx} for {}, got {:?}, expected {:?}
• unexpected dtype for {}, got {:?}, expected {dt:?}
• Reshape: at most one dimension of the target shape can be -1
• unsupported 'value' data-type {dt:?} for {}

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/333e9932ec8fe779. Report an issue: GitHub.