{"record":{"id":"6c820cb6f9979aae","repo":"huggingface/candle","slug":"shape-mismatch-k-and-v","errorCode":null,"errorMessage":"shape mismatch k {:?} and v {:?}","messagePattern":"shape mismatch k (.+?) and v (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-flash-attn/src/lib.rs","lineNumber":563,"sourceCode":"                \"flash-attn-varlen paged expects k/v tensors of rank 4 (k: {k_rank}, v: {v_rank})\"\n            )\n        }\n        if q_stride[q_rank - 1] != 1 {\n            candle::bail!(\"the last dim of q must be contiguous {q_stride:?}\")\n        }\n        if k_stride[k_rank - 1] != 1 {\n            candle::bail!(\"the last dim of k must be contiguous {k_stride:?}\")\n        }\n        if v_stride[v_rank - 1] != 1 {\n            candle::bail!(\"the last dim of v must be contiguous {v_stride:?}\")\n        }\n\n        let (total_q, num_heads, head_size_og) = q_l.shape().dims3()?;\n        let (num_heads_k, page_block_size) = if paged {\n            let (_, page_block_size, num_heads_k, k_head_size) = k_l.shape().dims4()?;\n            let expected_v = k_l.shape().dims4()?;\n            if expected_v != v_l.shape().dims4()? {\n                candle::bail!(\"shape mismatch k {:?} and v {:?}\", k_l.shape(), v_l.shape())\n            }\n            if k_head_size != head_size_og {\n                candle::bail!(\"shape mismatch q {:?} and k {:?}\", q_l.shape(), k_l.shape())\n            }\n            let Some(page_block_size_arg) = self.page_block_size else {\n                candle::bail!(\"paged flash-attn requires page_block_size\")\n            };\n            if page_block_size_arg != page_block_size {\n                candle::bail!(\n                    \"page_block_size {page_block_size_arg} does not match k shape {:?}\",\n                    k_l.shape()\n                )\n            }\n            if page_block_size % 32 != 0 {\n                candle::bail!(\n                    \"paged flash-attn requires page_block_size to be a multiple of 32 (got {page_block_size})\"\n                )\n            }","sourceCodeStart":545,"sourceCodeEnd":581,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-flash-attn/src/lib.rs#L545-L581","documentation":"In paged varlen mode, the k and v paged caches must have identical shapes: (num_blocks, page_block_size, num_heads_k, head_dim). The wrapper compares k's dims4 with v's dims4 and bails on any mismatch.","triggerScenarios":"Calling forward with block_table set and k_cache of shape e.g. (N, P, H, D) but v_cache of a different number of blocks, page size, head count, or head dim.","commonSituations":"Allocating K and V caches with different page_block_size or block counts; a refactoring that resized one cache but not the other; GQA changes applied to K but not V.","solutions":["Make v_cache match k_cache exactly in all four dims (num_blocks, page_block_size, num_heads_k, head_dim)","Re-allocate both caches from the same cache config at startup","Log/compare k.shape() and v.shape() before the call to find which dim differs"],"exampleFix":"// before\nlet k_cache = Tensor::zeros((n_blocks, 16, h_kv, d), ...)?;\nlet v_cache = Tensor::zeros((n_blocks, 32, h_kv, d), ...)?; // page size differs\n// after\nlet k_cache = Tensor::zeros((n_blocks, 16, h_kv, d), ...)?;\nlet v_cache = Tensor::zeros((n_blocks, 16, h_kv, d), ...)?;","handlingStrategy":"validation","validationCode":"fn check_kv_shapes_match(k: &candle_core::Tensor, v: &candle_core::Tensor) -> candle_core::Result<()> {\n    if k.shape() != v.shape() {\n        candle_core::bail!(\"kv cache shape mismatch: k {:?} vs v {:?}\", k.shape(), v.shape());\n    }\n    Ok(())\n}\n// call before forward in paged mode","typeGuard":"fn kv_shapes_match(k: &candle_core::Tensor, v: &candle_core::Tensor) -> bool {\n    k.shape() == v.shape()\n}","tryCatchPattern":"match attn.forward(&q, &k, &v, &sq, &sk, Some(&bt)) {\n    Ok(out) => out,\n    Err(e) if e.to_string().contains(\"shape mismatch k\") => {\n        candle_core::bail!(\"misconfigured paged cache: {} (k={:?} v={:?})\", e, k.shape(), v.shape())\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Allocate K and V caches from one shared cache-config struct","Assert k.shape() == v.shape() whenever a cache is resized","Apply GQA/head-count changes to both K and V caches simultaneously"],"tags":["cuda","flash-attention","paged-attention","shape-mismatch"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}