{"record":{"id":"0f0a80db0b182fd2","repo":"huggingface/candle","slug":"page-block-size-page-block-size-arg-does-not-mat","errorCode":null,"errorMessage":"page_block_size {page_block_size_arg} does not match k shape {:?}","messagePattern":"page_block_size (.+?) does not match k shape (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-flash-attn/src/lib.rs","lineNumber":572,"sourceCode":"        if v_stride[v_rank - 1] != 1 {\n            candle::bail!(\"the last dim of v must be contiguous {v_stride:?}\")\n        }\n\n        let (total_q, num_heads, head_size_og) = q_l.shape().dims3()?;\n        let (num_heads_k, page_block_size) = if paged {\n            let (_, page_block_size, num_heads_k, k_head_size) = k_l.shape().dims4()?;\n            let expected_v = k_l.shape().dims4()?;\n            if expected_v != v_l.shape().dims4()? {\n                candle::bail!(\"shape mismatch k {:?} and v {:?}\", k_l.shape(), v_l.shape())\n            }\n            if k_head_size != head_size_og {\n                candle::bail!(\"shape mismatch q {:?} and k {:?}\", q_l.shape(), k_l.shape())\n            }\n            let Some(page_block_size_arg) = self.page_block_size else {\n                candle::bail!(\"paged flash-attn requires page_block_size\")\n            };\n            if page_block_size_arg != page_block_size {\n                candle::bail!(\n                    \"page_block_size {page_block_size_arg} does not match k shape {:?}\",\n                    k_l.shape()\n                )\n            }\n            if page_block_size % 32 != 0 {\n                candle::bail!(\n                    \"paged flash-attn requires page_block_size to be a multiple of 32 (got {page_block_size})\"\n                )\n            }\n            if head_size_og > 512 {\n                candle::bail!(\"paged flash-attn supports head sizes up to 512 (got {head_size_og})\")\n            }\n            (num_heads_k, page_block_size_arg)\n        } else {\n            let (total_k, num_heads_k, _head_size_og) = k_l.shape().dims3()?;\n            let expected_kv = (total_k, num_heads_k, head_size_og);\n            if expected_kv != k_l.shape().dims3()? {\n                candle::bail!(\"shape mismatch q {:?} and k {:?}\", q_l.shape(), k_l.shape())","sourceCodeStart":554,"sourceCodeEnd":590,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-flash-attn/src/lib.rs#L554-L590","documentation":"When paged attention is active, the page_block_size passed to the builder must equal the page_block_size dimension (dim 1) of the 4D k cache. The wrapper compares them and bails on disagreement, since the kernel indexes the block table assuming that layout.","triggerScenarios":"Calling forward with block_table set, builder page_block_size = P1, but a k cache whose shape is (blocks, P2, Hk, D) with P1 != P2.","commonSituations":"Changing the KV cache block size in the runtime config without updating the attention builder; reusing a preallocated cache across models with different page sizes; copy-pasted configs where two block-size constants diverge.","solutions":["Use a single block-size constant for both the builder option and the cache allocation","Re-allocate the k/v caches with page_block_size matching the builder value","Read the actual page size from the cache shape and pass it to the builder instead of hardcoding"],"exampleFix":"// before\nlet attn = FlashAttentionVarLen::new(s, wl, wr, Some(16))?;\nlet k_cache = Tensor::zeros((n, 32, h_kv, d), ...)?; // 32 != 16\n// after\nlet page_block_size = 16;\nlet attn = FlashAttentionVarLen::new(s, wl, wr, Some(page_block_size))?;\nlet k_cache = Tensor::zeros((n, page_block_size, h_kv, d), ...)?;","handlingStrategy":"validation","validationCode":"fn check_page_block_size(builder_pbs: u32, k: &candle_core::Tensor) -> candle_core::Result<()> {\n    let k_dims = k.dims4()?;\n    if builder_pbs as usize != k_dims[1] {\n        candle_core::bail!(\n            \"page_block_size {} != k cache page dim {}\",\n            builder_pbs, k_dims[1]\n        );\n    }\n    Ok(())\n}","typeGuard":"fn page_block_size_matches(pbs: u32, k: &candle_core::Tensor) -> bool {\n    k.dims().get(1) == Some(&(pbs as usize))\n}","tryCatchPattern":"match attn.forward(&q, &k, &v, &sq, &sk, Some(&bt)) {\n    Ok(out) => out,\n    Err(e) if e.to_string().contains(\"does not match k shape\") => {\n        candle_core::bail!(\"cache page size vs builder page_block_size mismatch: {}\", e)\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Use one BLOCK_SIZE constant for both cache allocation and the attention builder","Read the page size from the cache tensor shape when constructing the attention struct","Add a startup assertion comparing cache dims and builder options"],"tags":["cuda","flash-attention","paged-attention","shape-mismatch","config"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}