{"record":{"id":"610a27c6c3a83025","repo":"huggingface/candle","slug":"input-rank-must-be-weight-rank","errorCode":null,"errorMessage":"input rank ({}) must be >= weight rank ({})","messagePattern":"input rank \\((.+?)\\) must be >= weight rank \\((.+?)\\)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-core/src/quantized/metal.rs","lineNumber":365,"sourceCode":"        storage: &MetalStorage,\n        layout: &crate::Layout,\n    ) -> Result<(MetalStorage, Shape)> {\n        use crate::MetalError;\n\n        if !layout.is_contiguous() {\n            crate::bail!(\"input tensor is not contiguous {layout:?}\")\n        }\n        let src_shape = layout.shape();\n        // self is transposed so n is first then k.\n        if src_shape.rank() < 2 {\n            crate::bail!(\"input tensor has only one dimension {layout:?}\")\n        }\n        let n = self_shape.dim(D::Minus2)?;\n        let k = self_shape.dim(D::Minus1)?;\n        let mut dst_shape = src_shape.dims().to_vec();\n\n        if src_shape.rank() < self_shape.rank() {\n            crate::bail!(\n                \"input rank ({}) must be >= weight rank ({})\",\n                src_shape.rank(),\n                self_shape.rank()\n            )\n        }\n\n        if src_shape.dim(D::Minus2)? == 1 {\n            return self.fwd_mv(self_shape, storage, layout);\n        }\n\n        let last_k = dst_shape.pop().unwrap();\n        if last_k != k {\n            crate::bail!(\"input tensor {layout:?} incompatible with {:?}\", self_shape)\n        }\n        dst_shape.push(n);\n        let dst_shape = Shape::from(dst_shape);\n        let device = storage.device().clone();\n        let dst = device","sourceCodeStart":347,"sourceCodeEnd":383,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-core/src/quantized/metal.rs#L347-L383","documentation":"Thrown by the Metal quantized matmul path when the input (src) tensor's rank is lower than the quantized weight tensor's rank. The kernel needs the input to be at least as high-dimensional as the weight so the last two dims line up for matrix multiplication.","triggerScenarios":"Calling QMatMul::fwd (or quantized matmul) on Metal with a weight of rank >= 3 (e.g. reshaped to [b, m, k]) while the input is rank 1 or 2.","commonSituations":"Batching a quantized layer where the weight was expanded/reshaped but the activation was not; passing a 1D vector into a rank-3 quantized weight matmul on the Metal backend.","solutions":["Reshape the input so its rank is >= the weight rank (e.g. unsqueeze batch dims) before the quantized matmul.","Flatten/reshape the weight to 2D [out, in] so a rank-2 input is sufficient.","Check the shapes right before the QMatMul call and align them.","If you don't need Metal, run the same op on CPU where the rank constraint is different."],"exampleFix":"// before\nlet y = qmatmul.forward(&x)?; // x rank 2, weight rank 3\n// after\nlet x = x.unsqueeze(0)?; // align input rank with weight rank\nlet y = qmatmul.forward(&x)?;","handlingStrategy":"validation","validationCode":"fn ensure_input_rank_ok(input_rank: usize, weight: &candle_core::quantized::QTensor) -> candle_core::Result<()> {\n    if input_rank < weight.rank() {\n        candle_core::bail!(\"input rank {} < weight rank {}\", input_rank, weight.rank());\n    }\n    Ok(())\n}","typeGuard":"fn input_rank_ok(input_rank: usize, weight_rank: usize) -> bool { input_rank >= weight_rank }","tryCatchPattern":null,"preventionTips":["Match input rank to weight rank with unsqueeze before quantized matmuls","Keep quantized weights 2D [out, in] unless batching requires more","Log shapes of weights and activations at model-load time"],"tags":["metal","quantized","shape-mismatch","matmul"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}