What causes "unexpected dim {idx} for {}, got {:?}, expected {:?}" in huggingface/candle?

Pad/truncate/reshape the input so each concrete dimension matches the declared shape shown in the error.

unexpected dim {idx} for {}, got {:?}, expected {:?}

Error message

unexpected dim {idx} for {}, got {:?}, expected {:?}

What it means

After rank validation, simple_eval_ compares each dimension pairwise. When a declared dim is a concrete DimValue and the corresponding tensor dim differs, this error is thrown with the index, the full declared shape, and the actual tensor shape. Dimensions declared as DimParam (symbolic) or unknown are skipped.

Source

Thrown at candle-onnx/src/eval.rs:300

            },
            type_ => bail!("unsupported input type {type_:?}"),
        };
        match &tensor_type.shape {
            None => continue,
            Some(shape) => {
                if shape.dim.len() != tensor.rank() {
                    bail!(
                        "unexpected rank for {}, got {:?}, expected {:?}",
                        input.name,
                        shape.dim,
                        tensor.shape()
                    )
                }
                for (idx, (d, &dim)) in shape.dim.iter().zip(tensor.dims().iter()).enumerate() {
                    match &d.value {
                        Some(onnx::tensor_shape_proto::dimension::Value::DimValue(v)) => {
                            if *v as usize != dim {
                                bail!(
                                    "unexpected dim {idx} for {}, got {:?}, expected {:?}",
                                    input.name,
                                    shape.dim,
                                    tensor.shape()
                                )
                            }
                        }
                        // We do not check equality constraints for the DimParam dimensions for now.
                        Some(onnx::tensor_shape_proto::dimension::Value::DimParam(_)) | None => (),
                    }
                }
            }
        };
        if dt != tensor.dtype() {
            bail!(
                "unexpected dtype for {}, got {:?}, expected {dt:?}",
                input.name,
                tensor.dtype()

View on GitHub (pinned to d5fee525bf)

Solutions

Pad/truncate/reshape the input so each concrete dimension matches the declared shape shown in the error.
Re-export the model with dynamic_axes for the dimensions you need to vary (batch, sequence length).
Check preprocessing (tokenizer max_length, image resize) matches the export configuration.
Split inputs into batches matching the exported static batch size, or re-export for multiple sizes.

Example fix

// before
let ids = tokenizer.encode(text, None)?; // arbitrary length
// after
let ids = tokenizer.encode(text, Some(512))?; // match declared fixed seq len 512
let ids = pad_to_len(&ids, 512);

Defensive patterns

Strategy: validation

Validate before calling

fn dims_match(model: &onnx::ModelProto, inputs: &HashMap<String, Value>) -> Vec<String> {
    let mut bad = vec![];
    if let Some(g) = &model.graph {
        for i in &g.input {
            if let (Some(t), Some(onnx::type_proto::Value::TensorType(tt))) =
                (inputs.get(&i.name), &i.r#type.value)
            {
                if let Some(s) = &tt.shape {
                    for (idx, (d, &dim)) in s.dim.iter().zip(t.dims()).enumerate() {
                        if let Some(onnx::tensor_shape_proto::dimension::Value::DimValue(v)) = &d.value {
                            if *v as usize != dim { bad.push(format!("{} dim {idx}: want {v}, got {dim}", i.name)); }
                        }
                    }
                }
            }
        }
    }
    bad
}

Try / catch

match simple_eval(&model, inputs) {
    Err(e) if e.to_string().starts_with("unexpected dim") => {
        anyhow::bail!("pad/truncate/resize input to the declared shape shown in the error")
    }
    r => r?,
}

Prevention

Pin tokenizer max_length and image sizes to the exported static dims
Export dynamic_axes for batch/sequence if shapes vary
Batch inputs to the exported static batch size
Pre-validate every concrete dim against model metadata before eval

When it happens

Trigger: simple_eval with an input tensor whose concrete dimension violates a fixed declared dim — e.g. seq_len 128 tensor for a model fixed at seq_len 512, or batch size 2 where the model declared batch 1.

Common situations: Variable-length inputs fed to a model exported with fixed sequence length; batch-size mismatch against a static-batch export; resized/spatial inputs not matching exported image size.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/bc8c75dce97879e4. Report an issue: GitHub.