What causes "unexpected shape for qkv {:?}" in huggingface/candle?

Verify hidden_size and num_heads in Config match the checkpoint so the qkv projection yields dim 2 == 3

unexpected shape for qkv {:?}

Error message

unexpected shape for qkv {:?}

What it means

apply_rotary_emb_qkv expects qkv shaped (b, seqlen, 3, heads, head_dim) where dim 2 is exactly 3 (q, k, v). When the third dimension is not 3, the tensor layout is not the packed QKV layout the MixFormer code expects, so it bails with the offending shape.

Source

Thrown at candle-transformers/src/models/mixformer.rs:174

        let inv_freq = Tensor::from_vec(inv_freq, (1, inv_freq_len), dev)?;
        let t = Tensor::arange(0u32, max_seq_len as u32, dev)?
            .to_dtype(DType::F32)?
            .reshape((max_seq_len, 1))?;
        let freqs = t.matmul(&inv_freq)?;
        Ok(Self {
            sin: freqs.sin()?.to_dtype(dtype)?,
            cos: freqs.cos()?.to_dtype(dtype)?,
        })
    }

    fn apply_rotary_emb_qkv(
        &self,
        qkv: &Tensor,
        seqlen_offset: usize,
    ) -> Result<(Tensor, Tensor, Tensor)> {
        let (_b_size, seqlen, three, _, _headdim) = qkv.dims5()?;
        if three != 3 {
            candle::bail!("unexpected shape for qkv {:?}", qkv.shape())
        }
        let (_rotary_seqlen, rotary_dim) = self.cos.dims2()?;
        let rotary_dim = rotary_dim * 2;
        let q_rot = qkv.i((.., .., 0, .., ..rotary_dim))?.contiguous()?;
        let q_pass = qkv.i((.., .., 0, .., rotary_dim..))?;
        let k_rot = qkv.i((.., .., 1, .., ..rotary_dim))?.contiguous()?;
        let k_pass = qkv.i((.., .., 1, .., rotary_dim..))?;
        let c = self.cos.narrow(0, seqlen_offset, seqlen)?;
        let s = self.sin.narrow(0, seqlen_offset, seqlen)?;
        let q_rot = candle_nn::rotary_emb::rope_thd(&q_rot, &c, &s)?;
        let k_rot = candle_nn::rotary_emb::rope_thd(&k_rot, &c, &s)?;
        let q = Tensor::cat(&[&q_rot, &q_pass], D::Minus1)?;
        let k = Tensor::cat(&[&k_rot, &k_pass], D::Minus1)?;
        let v = qkv.i((.., .., 2))?;
        Ok((q, k, v))
    }
}

View on GitHub (pinned to d5fee525bf)

Solutions

Verify hidden_size and num_heads in Config match the checkpoint so the qkv projection yields dim 2 == 3
Do not pre-split or reshape the qkv tensor before apply_rotary_emb_qkv
Inspect the printed shape in the error and fix the reshape/projection that produced it

Example fix

// before
let qkv = qkv.reshape((b, seq, heads * head_dim * 2, head_dim))?; // dim2 != 3
// after
let qkv = qkv.reshape((b, seq, 3, heads, head_dim))?; // packed q,k,v

Defensive patterns

Strategy: validation

Validate before calling

let (b, seq, three, heads, hd) = qkv.dims5()?;
if three != 3 {
    return Err(anyhow::anyhow!("qkv dim 2 must be 3, got {three}"));
}

Type guard

fn is_packed_qkv(qkv: &Tensor) -> candle::Result<bool> {
    Ok(qkv.dims5()?.2 == 3)
}

Try / catch

let (q, k, v) = match apply_rotary_emb_qkv(&qkv, offset) {
    Ok(v) => v,
    Err(e) if e.to_string().contains("unexpected shape for qkv") => {
        return Err(anyhow::anyhow!("check hidden_size/num_heads; qkv must be (b, seq, 3, heads, hd)"))
    }
    Err(e) => return Err(e.into()),
};

Prevention

Assert hidden_size == num_heads * head_dim in the config
Never reshape or pre-split the packed qkv tensor before rotary embedding
Print tensor shapes when wiring new model variants into MixFormer

When it happens

Trigger: Calling MixFormer forward paths (via apply_rotary_emb_qkv) with a qkv tensor whose dim 2 != 3, usually caused by a wrong head-count/hidden-size config producing a mis-shaped projection, or feeding a pre-split q/k/v tensor into the packed path.

Common situations: Config mismatch between hidden_size/num_heads and the checkpoint weights; manually reshaping or splitting qkv before calling; using a model variant that packs attention differently.

Understand the failure class

Background: Tensor shape mismatch errors ("must have shape", "expected shape ... got ..."): when tensor dimensions disagree with what an op or layer was told to expect — this error's family across 6 libraries.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/3c8c15bc736e13b3. Report an issue: GitHub.