{"record":{"id":"c327fa4e6e388ba2","repo":"huggingface/candle","slug":"unexpected-lhs-length-mkn","errorCode":null,"errorMessage":"unexpected lhs length {} {mkn:?}","messagePattern":"unexpected lhs length (.+?) (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-core/src/quantized/k_quants.rs","lineNumber":2676,"sourceCode":"                        std::ptr::copy_nonoverlapping(results.as_ptr(), dst_ptr.add(g * 8), 8);\n                    }\n                }\n            });\n        }\n\n        Ok(())\n    })\n}\n\npub fn matmul_f16<T: GgmlType>(\n    mkn: (usize, usize, usize),\n    lhs: &[f16],\n    rhs_t: &[T],\n    dst: &mut [f16],\n) -> Result<()> {\n    let (m, k, n) = mkn;\n    if m * k != lhs.len() {\n        crate::bail!(\"unexpected lhs length {} {mkn:?}\", lhs.len());\n    }\n\n    let k_in_lhs_blocks = k.div_ceil(T::BLCK_SIZE);\n    let k_in_rhs_blocks = k.div_ceil(T::VecDotType::BLCK_SIZE);\n    let mut lhs_b = vec![T::VecDotType::zeros(); m * k_in_lhs_blocks];\n    for row_idx in 0..m {\n        let lhs_b = &mut lhs_b[row_idx * k_in_lhs_blocks..(row_idx + 1) * k_in_lhs_blocks];\n        let lhs = &lhs[row_idx * k..(row_idx + 1) * k];\n        let lhs_f32: Vec<_> = lhs.iter().map(|&x| x.to_f32()).collect();\n        T::VecDotType::from_float(&lhs_f32, lhs_b);\n    }\n    let lhs_b = lhs_b.as_slice();\n\n    for row_idx in 0..m {\n        let lhs_row = &lhs_b[row_idx * k_in_lhs_blocks..(row_idx + 1) * k_in_lhs_blocks];\n        let dst_row = &mut dst[row_idx * n..(row_idx + 1) * n];\n\n        for (col_idx, dst) in dst_row.iter_mut().enumerate() {","sourceCodeStart":2658,"sourceCodeEnd":2694,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-core/src/quantized/k_quants.rs#L2658-L2694","documentation":"This quantized matmul helper takes the logical (m, k, n) shape and the LHS slice; it verifies that m * k equals the LHS buffer length before dequantizing into blocks. A mismatch means the caller supplied a buffer whose length does not match the declared shape, so proceeding would read out of bounds.","triggerScenarios":"Calling a quantized matmul routine (e.g. matmul on Q8_0/Q4K etc. via vec_dot) with an lhs slice whose length differs from m*k — typically from slicing errors, mismatched tensor shapes between weights and activations, or a candle-internal bug.","commonSituations":"Model weight shapes incompatible with the input (wrong model config, e.g. wrong hidden size); mixing tensors from different model checkpoints; constructing views/reshapes with wrong dims before a quantized op.","solutions":["Verify input tensor shapes match the model's expected hidden size (k) and batch/rows (m)","Check that both operands come from the same model with matching config (hidden_dim, num_heads)","Ensure reshape/view calls preserve total element counts before quantized ops; update candle if reproducible on standard models"],"exampleFix":"// before\nlet x = xs.reshape((batch, wrong_hidden))?; // mismatched with weight k\nlet y = qmatmul.forward(&x)?;\n// after\nlet x = xs.reshape((batch, hidden_dim))?; // must satisfy batch * hidden_dim == x.len()\nassert_eq!(batch * hidden_dim, x.elem_count());\nlet y = qmatmul.forward(&x)?;","handlingStrategy":"validation","validationCode":"let (b, s) = xs.dims2()?;\nassert_eq!(s, hidden_dim, \"input last dim {} != model hidden {}\", s, hidden_dim);\nassert_eq!(b * s, xs.elem_count());","typeGuard":null,"tryCatchPattern":"match qmatmul.forward(&xs) {\n    Ok(y) => y,\n    Err(e) if e.to_string().contains(\"unexpected lhs length\") => {\n        eprintln!(\"input shape {:?} incompatible with weight shape {:?}\", xs.dims(), qmatmul.dims());\n        return Err(e.into());\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Check input hidden size against the model config before quantized matmuls","Keep operands from the same checkpoint/config","Assert total element count matches m*k before low-level quantized calls"],"tags":["quantization","matmul","shape-mismatch"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}