huggingface/candle · error

unexpected lhs length {} {mkn:?}

Error message

unexpected lhs length {} {mkn:?}

What it means

This quantized matmul helper takes the logical (m, k, n) shape and the LHS slice; it verifies that m * k equals the LHS buffer length before dequantizing into blocks. A mismatch means the caller supplied a buffer whose length does not match the declared shape, so proceeding would read out of bounds.

Source

Thrown at candle-core/src/quantized/k_quants.rs:2676

                        std::ptr::copy_nonoverlapping(results.as_ptr(), dst_ptr.add(g * 8), 8);
                    }
                }
            });
        }

        Ok(())
    })
}

pub fn matmul_f16<T: GgmlType>(
    mkn: (usize, usize, usize),
    lhs: &[f16],
    rhs_t: &[T],
    dst: &mut [f16],
) -> Result<()> {
    let (m, k, n) = mkn;
    if m * k != lhs.len() {
        crate::bail!("unexpected lhs length {} {mkn:?}", lhs.len());
    }

    let k_in_lhs_blocks = k.div_ceil(T::BLCK_SIZE);
    let k_in_rhs_blocks = k.div_ceil(T::VecDotType::BLCK_SIZE);
    let mut lhs_b = vec![T::VecDotType::zeros(); m * k_in_lhs_blocks];
    for row_idx in 0..m {
        let lhs_b = &mut lhs_b[row_idx * k_in_lhs_blocks..(row_idx + 1) * k_in_lhs_blocks];
        let lhs = &lhs[row_idx * k..(row_idx + 1) * k];
        let lhs_f32: Vec<_> = lhs.iter().map(|&x| x.to_f32()).collect();
        T::VecDotType::from_float(&lhs_f32, lhs_b);
    }
    let lhs_b = lhs_b.as_slice();

    for row_idx in 0..m {
        let lhs_row = &lhs_b[row_idx * k_in_lhs_blocks..(row_idx + 1) * k_in_lhs_blocks];
        let dst_row = &mut dst[row_idx * n..(row_idx + 1) * n];

        for (col_idx, dst) in dst_row.iter_mut().enumerate() {

View on GitHub (pinned to d5fee525bf)

Solutions

1. Verify input tensor shapes match the model's expected hidden size (k) and batch/rows (m)
2. Check that both operands come from the same model with matching config (hidden_dim, num_heads)
3. Ensure reshape/view calls preserve total element counts before quantized ops; update candle if reproducible on standard models

Example fix

// before
let x = xs.reshape((batch, wrong_hidden))?; // mismatched with weight k
let y = qmatmul.forward(&x)?;
// after
let x = xs.reshape((batch, hidden_dim))?; // must satisfy batch * hidden_dim == x.len()
assert_eq!(batch * hidden_dim, x.elem_count());
let y = qmatmul.forward(&x)?;

let (b, s) = xs.dims2()?;
assert_eq!(s, hidden_dim, "input last dim {} != model hidden {}", s, hidden_dim);
assert_eq!(b * s, xs.elem_count());

match qmatmul.forward(&xs) {
    Ok(y) => y,
    Err(e) if e.to_string().contains("unexpected lhs length") => {
        eprintln!("input shape {:?} incompatible with weight shape {:?}", xs.dims(), qmatmul.dims());
        return Err(e.into());
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling a quantized matmul routine (e.g. matmul on Q8_0/Q4K etc. via vec_dot) with an lhs slice whose length differs from m*k — typically from slicing errors, mismatched tensor shapes between weights and activations, or a candle-internal bug.

Common situations: Model weight shapes incompatible with the input (wrong model config, e.g. wrong hidden size); mixing tensors from different model checkpoints; constructing views/reshapes with wrong dims before a quantized op.

Understand the failure class

Background: Tensor shape mismatch errors ("must have shape", "expected shape ... got ..."): when tensor dimensions disagree with what an op or layer was told to expect — this error's family across 6 libraries.

Related errors

• mismatch on matmul dim {self_shape:?} {:?}
• unexpected rhs shape in dmmv {:?}
• unexpected shape for input {s:?}
• input rank ({}) must be >= weight rank ({})
• backward not supported for non uniform upscaling factors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/c327fa4e6e388ba2. Report an issue: GitHub.