{"record":{"id":"6118096bc34ec7f4","repo":"huggingface/candle","slug":"flash-attn-varlen-paged-expects-k-v-tensors-of-ran","errorCode":null,"errorMessage":"flash-attn-varlen paged expects k/v tensors of rank 4 (k: {k_rank}, v: {v_rank})","messagePattern":"flash-attn-varlen paged expects k/v tensors of rank 4 \\(k: (.+?), v: (.+?)\\)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-flash-attn/src/lib.rs","lineNumber":544,"sourceCode":"\n        let q_stride = q_l.stride();\n        let k_stride = k_l.stride();\n        let v_stride = v_l.stride();\n        let o_stride = out_l.stride();\n\n        let q_rank = q_stride.len();\n        let k_rank = k_stride.len();\n        let v_rank = v_stride.len();\n        let o_rank = o_stride.len();\n\n        let paged = block_table.is_some();\n        if q_rank != 3 || (!paged && k_rank != 3) || (!paged && v_rank != 3) {\n            candle::bail!(\n                \"flash-attn-varlen expects input tensors of rank 3 (q: {q_rank}, k: {k_rank}, v: {v_rank}\"\n            )\n        }\n        if paged && (k_rank != 4 || v_rank != 4) {\n            candle::bail!(\n                \"flash-attn-varlen paged expects k/v tensors of rank 4 (k: {k_rank}, v: {v_rank})\"\n            )\n        }\n        if q_stride[q_rank - 1] != 1 {\n            candle::bail!(\"the last dim of q must be contiguous {q_stride:?}\")\n        }\n        if k_stride[k_rank - 1] != 1 {\n            candle::bail!(\"the last dim of k must be contiguous {k_stride:?}\")\n        }\n        if v_stride[v_rank - 1] != 1 {\n            candle::bail!(\"the last dim of v must be contiguous {v_stride:?}\")\n        }\n\n        let (total_q, num_heads, head_size_og) = q_l.shape().dims3()?;\n        let (num_heads_k, page_block_size) = if paged {\n            let (_, page_block_size, num_heads_k, k_head_size) = k_l.shape().dims4()?;\n            let expected_v = k_l.shape().dims4()?;\n            if expected_v != v_l.shape().dims4()? {","sourceCodeStart":526,"sourceCodeEnd":562,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-flash-attn/src/lib.rs#L526-L562","documentation":"In paged varlen flash-attention mode (a block_table was supplied), k and v must be rank-4 PagedCache tensors shaped (num_blocks, page_block_size, num_heads_k, head_dim), while q stays rank 3. The wrapper bails if either k or v is not rank 4 when paged attention is active.","triggerScenarios":"Calling cuda_fwd_t with block_table = Some(...) but passing k or v with rank 3 (plain contiguous KV cache) or any other rank, instead of a 4D paged KV cache.","commonSituations":"Enabling paged attention (e.g. switching to a vLLM-style block KV cache) but forgetting to restructure the KV cache into blocks; mixing paged attention with non-paged cache tensors left over from the previous implementation.","solutions":["Reshape/allocate k and v as rank-4 paged caches: (num_blocks, page_block_size, num_heads_k, head_dim)","If you do not intend paged attention, pass block_table = None so rank-3 k/v are accepted","Ensure the paged cache tensor is contiguous and strided as a proper 4D tensor (not a flattened view without shape metadata)"],"exampleFix":"// before (paged mode with rank-3 kv)\nlet out = fwd.forward(q, &k_cache, &v_cache, &seqlens_q, &seqlens_k, Some(&block_table))?;\n// after\nlet k_cache = k_cache.reshape((num_blocks, page_block_size, h_kv, d))?;\nlet v_cache = v_cache.reshape((num_blocks, page_block_size, h_kv, d))?;\nlet out = fwd.forward(q, &k_cache, &v_cache, &seqlens_q, &seqlens_k, Some(&block_table))?;","handlingStrategy":"validation","validationCode":"fn check_paged_kv(k: &candle_core::Tensor, v: &candle_core::Tensor) -> candle_core::Result<()> {\n    for (name, t) in [(\"k\", k), (\"v\", v)] {\n        if t.dims().len() != 4 {\n            candle_core::bail!(\"paged {name} cache must be rank 4 (blocks, page, heads, head_dim), got {:?}\", t.dims());\n        }\n    }\n    Ok(())\n}\n// call before forward when block_table.is_some()","typeGuard":"fn is_rank4(t: &candle_core::Tensor) -> bool { t.dims().len() == 4 }","tryCatchPattern":"match attn.forward(&q, &k, &v, &sq, &sk, block_table) {\n    Ok(out) => out,\n    Err(e) if e.to_string().contains(\"paged expects k/v tensors of rank 4\") => {\n        candle_core::bail!(\"KV cache not allocated as paged (blocks, page, heads, head_dim): {}\", e)\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Allocate K/V caches through a single PagedCache helper that enforces rank 4","Gate paged code paths with an explicit config flag and assert cache rank there","Never mix paged block_table with plain contiguous KV caches"],"tags":["cuda","flash-attention","paged-attention","tensor-shape"],"backgroundTag":"tensor-rank-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}