{"record":{"id":"97c5af48a035d958","repo":"huggingface/candle","slug":"unexpected-shape-for-input-s","errorCode":null,"errorMessage":"unexpected shape for input {s:?}","messagePattern":"unexpected shape for input (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-core/src/quantized/cuda.rs","lineNumber":952,"sourceCode":"        };\n        let mut out_shape = rhs_l.shape().dims().to_vec();\n        out_shape.pop();\n        out_shape.push(nrows);\n        Ok((out, out_shape.into()))\n    }\n\n    fn dequantize_matmul(\n        &self,\n        self_shape: &crate::Shape,\n        storage: &CudaStorage,\n        layout: &crate::Layout,\n    ) -> Result<(CudaStorage, crate::Shape)> {\n        use crate::backend::BackendStorage;\n        let (n, k) = self_shape.dims2()?;\n        let (b, m, k2) = match layout.shape().dims() {\n            &[b, m, k2] => (b, m, k2),\n            &[m, k2] => (1, m, k2),\n            s => crate::bail!(\"unexpected shape for input {s:?}\"),\n        };\n        if k2 != k {\n            crate::bail!(\"mismatch on matmul dim {self_shape:?} {:?}\", layout.shape())\n        }\n\n        let out = if FORCE_DMMV.load(std::sync::atomic::Ordering::Relaxed) {\n            let data_f32 = self.dequantize(n * k)?;\n            let rhs_l = crate::Layout::new((k, n).into(), vec![1, k], 0).broadcast_as((b, k, n))?;\n            storage.matmul(&data_f32, (b, m, n, k), layout, &rhs_l)?\n        } else {\n            let storage = storage.as_cuda_slice::<f32>()?;\n            let storage = match layout.contiguous_offsets() {\n                Some((o1, o2)) => storage.slice(o1..o2),\n                None => Err(crate::Error::RequiresContiguous {\n                    op: \"quantized-matmul\",\n                }\n                .bt())?,\n            };","sourceCodeStart":934,"sourceCodeEnd":970,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-core/src/quantized/cuda.rs#L934-L970","documentation":"dequantize_matmul (the full matmul path) accepts an input of shape [b, m, k2] or [m, k2] (treated as batch 1). Any other shape — rank 1, rank 4+, or non-standard patterns — triggers this bail printing the actual shape. It mirrors the dmmv rank check for the non-vector path.","triggerScenarios":"Calling QTensor::fwd with an input of rank != 2/3 — e.g. a [k] vector, a [b, heads, m, k] attention tensor, or a [b, s, p, k] tensor from an earlier reshape.","commonSituations":"Passing multi-head attention tensors (rank 4) directly to a quantized layer without flattening heads; feeding a raw 1-D embedding; pipeline reshaping bugs where an extra dim remains.","solutions":["Flatten to 3-D first: let x = x.flatten_from(1)?; or reshape to [b, m, k].","Unsqueeze a batch dim for rank-1 input.","Fold head/extra dims into the batch dimension before the matmul.","Verify input rank with x.dims().len() before calling the quantized op."],"exampleFix":"// before\nlet out = qw.forward(&attn)?; // [b, heads, m, k]\n// after\nlet (b, h, m, k) = attn.dims4()?;\nlet out = qw.forward(&attn.reshape((b * h, m, k))?)?;","handlingStrategy":"validation","validationCode":"let dims = input.dims();\nif !(dims.len() == 2 || dims.len() == 3) {\n    let input = if dims.len() == 1 { input.unsqueeze(0)? }\n        else { input.flatten_to(2)? }; // fold extra dims into batch\n}\nqw.forward(&input)?;","typeGuard":null,"tryCatchPattern":"match qw.forward(&input) {\n    Err(e) if e.to_string().contains(\"unexpected shape for input\") => {\n        let x = if input.dims().len() == 1 { input.unsqueeze(0)? } else { input.flatten_to(2)? };\n        qw.forward(&x)?\n    }\n    r => r?,\n}","preventionTips":["Reshape rank-4 attention tensors to [b*heads, m, k] before quantized matmuls","Unsqueeze 1-D inputs to add a batch dim","Check rank with dims().len() in a helper before any quantized op"],"tags":["cuda","quantization","matmul","shape","rank"],"backgroundTag":"shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}