{"record":{"id":"ac9ab66be6295273","repo":"huggingface/candle","slug":"different-inner-dimensions-in-broadcast-matmul-lh","errorCode":null,"errorMessage":"different inner dimensions in broadcast matmul {lhs:?} {rhs:?}","messagePattern":"different inner dimensions in broadcast matmul (.+?) (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-core/src/shape.rs","lineNumber":239,"sourceCode":"                    op,\n                }\n                .bt())?\n            }\n        }\n        Ok(Shape::from(bcast_dims))\n    }\n\n    pub(crate) fn broadcast_shape_matmul(&self, rhs: &Self) -> Result<(Shape, Shape)> {\n        let lhs = self;\n        let lhs_dims = lhs.dims();\n        let rhs_dims = rhs.dims();\n        if lhs_dims.len() < 2 || rhs_dims.len() < 2 {\n            crate::bail!(\"only 2d matrixes are supported {lhs:?} {rhs:?}\")\n        }\n        let (m, lhs_k) = (lhs_dims[lhs_dims.len() - 2], lhs_dims[lhs_dims.len() - 1]);\n        let (rhs_k, n) = (rhs_dims[rhs_dims.len() - 2], rhs_dims[rhs_dims.len() - 1]);\n        if lhs_k != rhs_k {\n            crate::bail!(\"different inner dimensions in broadcast matmul {lhs:?} {rhs:?}\")\n        }\n\n        let lhs_b = Self::from(&lhs_dims[..lhs_dims.len() - 2]);\n        let rhs_b = Self::from(&rhs_dims[..rhs_dims.len() - 2]);\n        let bcast = lhs_b.broadcast_shape_binary_op(&rhs_b, \"broadcast_matmul\")?;\n        let bcast_dims = bcast.dims();\n\n        let bcast_lhs = [bcast_dims, &[m, lhs_k]].concat();\n        let bcast_rhs = [bcast_dims, &[rhs_k, n]].concat();\n        Ok((Shape::from(bcast_lhs), Shape::from(bcast_rhs)))\n    }\n}\n\npub trait Dim {\n    fn to_index(&self, shape: &Shape, op: &'static str) -> Result<usize>;\n    fn to_index_plus_one(&self, shape: &Shape, op: &'static str) -> Result<usize>;\n}\n","sourceCodeStart":221,"sourceCodeEnd":257,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-core/src/shape.rs#L221-L257","documentation":"After extracting the trailing 2x2 matrix dims, broadcast_shape_matmul checks that the inner (contraction) dimensions match: lhs last dim must equal rhs second-to-last dim. If lhs_k != rhs_k the matmul is mathematically undefined and the library bails, echoing both shapes.","triggerScenarios":"Calling matmul/broadcast_matmul with shapes like (m, k1) x (k2, n) where k1 != k2, or batched variants where the last two dims are incompatible.","commonSituations":"Mixing layers with wrong hidden sizes (e.g. 768 vs 1024 weights); transposing one operand incorrectly (forgot .t() or applied it twice); loading mismatched checkpoints; reshape mistakes that alter the K dimension.","solutions":["Fix the contraction dimension: transpose the RHS (or LHS) so inner dims match, e.g. x.matmul(&w.t()?)","Verify weight shapes match the model config's hidden size; reload the correct checkpoint","Print both .dims() and align reshapes so lhs.dims()[-1] == rhs.dims()[-2]"],"exampleFix":"// before\nlet y = x.matmul(&w)?;        // x: (b, 768), w: (1024, 768)\n// after\nlet y = x.matmul(&w.t()?)?;   // w.t(): (768, 1024)","handlingStrategy":"validation","validationCode":"let (l, r) = (lhs.dims(), rhs.dims());\nif l[l.len()-1] != r[r.len()-2] {\n    return Err(anyhow::anyhow!(\"matmul inner dims differ: {:?} x {:?}\", l, r));\n}","typeGuard":null,"tryCatchPattern":"let y = lhs.matmul(&rhs).map_err(|e| {\n    if e.to_string().contains(\"inner dimensions\") {\n        anyhow::anyhow!(\"check weight transpose/hidden size: {:?} vs {:?}\", lhs.dims(), rhs.dims())\n    } else { e.into() }\n})?;","preventionTips":["Verify lhs.dims()[-1] == rhs.dims()[-2] before every matmul","Check checkpoint weight shapes against model config hidden sizes","Remember candle matmul does not transpose: add .t() explicitly where needed"],"tags":["candle","matmul","shape","dimension-mismatch"],"backgroundTag":"matmul-dimension-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}