{"record":{"id":"3c8c15bc736e13b3","repo":"huggingface/candle","slug":"unexpected-shape-for-qkv","errorCode":null,"errorMessage":"unexpected shape for qkv {:?}","messagePattern":"unexpected shape for qkv (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/mixformer.rs","lineNumber":174,"sourceCode":"        let inv_freq = Tensor::from_vec(inv_freq, (1, inv_freq_len), dev)?;\n        let t = Tensor::arange(0u32, max_seq_len as u32, dev)?\n            .to_dtype(DType::F32)?\n            .reshape((max_seq_len, 1))?;\n        let freqs = t.matmul(&inv_freq)?;\n        Ok(Self {\n            sin: freqs.sin()?.to_dtype(dtype)?,\n            cos: freqs.cos()?.to_dtype(dtype)?,\n        })\n    }\n\n    fn apply_rotary_emb_qkv(\n        &self,\n        qkv: &Tensor,\n        seqlen_offset: usize,\n    ) -> Result<(Tensor, Tensor, Tensor)> {\n        let (_b_size, seqlen, three, _, _headdim) = qkv.dims5()?;\n        if three != 3 {\n            candle::bail!(\"unexpected shape for qkv {:?}\", qkv.shape())\n        }\n        let (_rotary_seqlen, rotary_dim) = self.cos.dims2()?;\n        let rotary_dim = rotary_dim * 2;\n        let q_rot = qkv.i((.., .., 0, .., ..rotary_dim))?.contiguous()?;\n        let q_pass = qkv.i((.., .., 0, .., rotary_dim..))?;\n        let k_rot = qkv.i((.., .., 1, .., ..rotary_dim))?.contiguous()?;\n        let k_pass = qkv.i((.., .., 1, .., rotary_dim..))?;\n        let c = self.cos.narrow(0, seqlen_offset, seqlen)?;\n        let s = self.sin.narrow(0, seqlen_offset, seqlen)?;\n        let q_rot = candle_nn::rotary_emb::rope_thd(&q_rot, &c, &s)?;\n        let k_rot = candle_nn::rotary_emb::rope_thd(&k_rot, &c, &s)?;\n        let q = Tensor::cat(&[&q_rot, &q_pass], D::Minus1)?;\n        let k = Tensor::cat(&[&k_rot, &k_pass], D::Minus1)?;\n        let v = qkv.i((.., .., 2))?;\n        Ok((q, k, v))\n    }\n}\n","sourceCodeStart":156,"sourceCodeEnd":192,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/mixformer.rs#L156-L192","documentation":"apply_rotary_emb_qkv expects qkv shaped (b, seqlen, 3, heads, head_dim) where dim 2 is exactly 3 (q, k, v). When the third dimension is not 3, the tensor layout is not the packed QKV layout the MixFormer code expects, so it bails with the offending shape.","triggerScenarios":"Calling MixFormer forward paths (via apply_rotary_emb_qkv) with a qkv tensor whose dim 2 != 3, usually caused by a wrong head-count/hidden-size config producing a mis-shaped projection, or feeding a pre-split q/k/v tensor into the packed path.","commonSituations":"Config mismatch between hidden_size/num_heads and the checkpoint weights; manually reshaping or splitting qkv before calling; using a model variant that packs attention differently.","solutions":["Verify hidden_size and num_heads in Config match the checkpoint so the qkv projection yields dim 2 == 3","Do not pre-split or reshape the qkv tensor before apply_rotary_emb_qkv","Inspect the printed shape in the error and fix the reshape/projection that produced it"],"exampleFix":"// before\nlet qkv = qkv.reshape((b, seq, heads * head_dim * 2, head_dim))?; // dim2 != 3\n// after\nlet qkv = qkv.reshape((b, seq, 3, heads, head_dim))?; // packed q,k,v","handlingStrategy":"validation","validationCode":"let (b, seq, three, heads, hd) = qkv.dims5()?;\nif three != 3 {\n    return Err(anyhow::anyhow!(\"qkv dim 2 must be 3, got {three}\"));\n}","typeGuard":"fn is_packed_qkv(qkv: &Tensor) -> candle::Result<bool> {\n    Ok(qkv.dims5()?.2 == 3)\n}","tryCatchPattern":"let (q, k, v) = match apply_rotary_emb_qkv(&qkv, offset) {\n    Ok(v) => v,\n    Err(e) if e.to_string().contains(\"unexpected shape for qkv\") => {\n        return Err(anyhow::anyhow!(\"check hidden_size/num_heads; qkv must be (b, seq, 3, heads, hd)\"))\n    }\n    Err(e) => return Err(e.into()),\n};","preventionTips":["Assert hidden_size == num_heads * head_dim in the config","Never reshape or pre-split the packed qkv tensor before rotary embedding","Print tensor shapes when wiring new model variants into MixFormer"],"tags":["rust","candle","mixformer","tensor-shape","rotary-embedding"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}