huggingface/candle · error

unexpected lhs length

Error message

unexpected lhs length {} {mkn:?}

What it means

This quantized matmul helper takes the logical (m, k, n) shape and the LHS slice; it verifies that m * k equals the LHS buffer length before dequantizing into blocks. A mismatch means the caller supplied a buffer whose length does not match the declared shape, so proceeding would read out of bounds.

Solutions

1. Verify input tensor shapes match the model's expected hidden size (k) and batch/rows (m)
2. Check that both operands come from the same model with matching config (hidden_dim, num_heads)
3. Ensure reshape/view calls preserve total element counts before quantized ops; update candle if reproducible on standard models

Example fix

// before
let x = xs.reshape((batch, wrong_hidden))?; // mismatched with weight k
let y = qmatmul.forward(&x)?;
// after
let x = xs.reshape((batch, hidden_dim))?; // must satisfy batch * hidden_dim == x.len()
assert_eq!(batch * hidden_dim, x.elem_count());
let y = qmatmul.forward(&x)?;

let (b, s) = xs.dims2()?;
assert_eq!(s, hidden_dim, "input last dim {} != model hidden {}", s, hidden_dim);
assert_eq!(b * s, xs.elem_count());

match qmatmul.forward(&xs) {
    Ok(y) => y,
    Err(e) if e.to_string().contains("unexpected lhs length") => {
        eprintln!("input shape {:?} incompatible with weight shape {:?}", xs.dims(), qmatmul.dims());
        return Err(e.into());
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling a quantized matmul routine (e.g. matmul on Q8_0/Q4K etc. via vec_dot) with an lhs slice whose length differs from m*k — typically from slicing errors, mismatched tensor shapes between weights and activations, or a candle-internal bug.

Common situations: Model weight shapes incompatible with the input (wrong model config, e.g. wrong hidden size); mixing tensors from different model checkpoints; constructing views/reshapes with wrong dims before a quantized op.

Understand the failure class

Background: Tensor shape mismatch errors ("must have shape", "expected shape ... got ..."): when tensor dimensions disagree with what an op or layer was told to expect — this error's family across 6 libraries.

Related errors

• mismatch on matmul dim
• input rank ( ) must be >= weight rank ( )
• unexpected rhs shape in dmmv
• unexpected shape for input
• backward not supported for non uniform upscaling factors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/c327fa4e6e388ba2. Report an issue: GitHub.

Appendix: source

Thrown at candle-core/src/quantized/k_quants.rs:2676

                        std::ptr::copy_nonoverlapping(results.as_ptr(), dst_ptr.add(g * 8), 8);
                    }
                }
            });
        }

        Ok(())
    })
}

pub fn matmul_f16<T: GgmlType>(
    mkn: (usize, usize, usize),
    lhs: &[f16],
    rhs_t: &[T],
    dst: &mut [f16],
) -> Result<()> {
    let (m, k, n) = mkn;
    if m * k != lhs.len() {
        crate::bail!("unexpected lhs length {} {mkn:?}", lhs.len());
    }

    let k_in_lhs_blocks = k.div_ceil(T::BLCK_SIZE);
    let k_in_rhs_blocks = k.div_ceil(T::VecDotType::BLCK_SIZE);
    let mut lhs_b = vec![T::VecDotType::zeros(); m * k_in_lhs_blocks];
    for row_idx in 0..m {
        let lhs_b = &mut lhs_b[row_idx * k_in_lhs_blocks..(row_idx + 1) * k_in_lhs_blocks];
        let lhs = &lhs[row_idx * k..(row_idx + 1) * k];
        let lhs_f32: Vec<_> = lhs.iter().map(|&x| x.to_f32()).collect();
        T::VecDotType::from_float(&lhs_f32, lhs_b);
    }
    let lhs_b = lhs_b.as_slice();

    for row_idx in 0..m {
        let lhs_row = &lhs_b[row_idx * k_in_lhs_blocks..(row_idx + 1) * k_in_lhs_blocks];
        let dst_row = &mut dst[row_idx * n..(row_idx + 1) * n];

        for (col_idx, dst) in dst_row.iter_mut().enumerate() {

View on GitHub (pinned to d5fee525bf)