{"record":{"id":"fedfd9903c238f7b","repo":"huggingface/candle","slug":"paged-flash-attn-requires-page-block-size-to-be-a","errorCode":null,"errorMessage":"paged flash-attn requires page_block_size to be a multiple of 32 (got {page_block_size})","messagePattern":"paged flash-attn requires page_block_size to be a multiple of 32 \\(got (.+?)\\)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-flash-attn/src/lib.rs","lineNumber":578,"sourceCode":"            let (_, page_block_size, num_heads_k, k_head_size) = k_l.shape().dims4()?;\n            let expected_v = k_l.shape().dims4()?;\n            if expected_v != v_l.shape().dims4()? {\n                candle::bail!(\"shape mismatch k {:?} and v {:?}\", k_l.shape(), v_l.shape())\n            }\n            if k_head_size != head_size_og {\n                candle::bail!(\"shape mismatch q {:?} and k {:?}\", q_l.shape(), k_l.shape())\n            }\n            let Some(page_block_size_arg) = self.page_block_size else {\n                candle::bail!(\"paged flash-attn requires page_block_size\")\n            };\n            if page_block_size_arg != page_block_size {\n                candle::bail!(\n                    \"page_block_size {page_block_size_arg} does not match k shape {:?}\",\n                    k_l.shape()\n                )\n            }\n            if page_block_size % 32 != 0 {\n                candle::bail!(\n                    \"paged flash-attn requires page_block_size to be a multiple of 32 (got {page_block_size})\"\n                )\n            }\n            if head_size_og > 512 {\n                candle::bail!(\"paged flash-attn supports head sizes up to 512 (got {head_size_og})\")\n            }\n            (num_heads_k, page_block_size_arg)\n        } else {\n            let (total_k, num_heads_k, _head_size_og) = k_l.shape().dims3()?;\n            let expected_kv = (total_k, num_heads_k, head_size_og);\n            if expected_kv != k_l.shape().dims3()? {\n                candle::bail!(\"shape mismatch q {:?} and k {:?}\", q_l.shape(), k_l.shape())\n            }\n            if expected_kv != v_l.shape().dims3()? {\n                candle::bail!(\"shape mismatch q {:?} and v {:?}\", q_l.shape(), v_l.shape())\n            }\n            (num_heads_k, 0)\n        };","sourceCodeStart":560,"sourceCodeEnd":596,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-flash-attn/src/lib.rs#L560-L596","documentation":"The paged flash-attention CUDA kernel only supports page (block) sizes that are a multiple of 32. After validating the builder value against k's shape, the wrapper checks page_block_size % 32 == 0 and bails otherwise.","triggerScenarios":"Calling forward in paged mode where the validated page_block_size (from k's shape, which equals the builder value) is e.g. 16, 48, or any non-multiple of 32.","commonSituations":"Choosing a small page size (e.g. 16) to reduce memory fragmentation in a paged KV cache; adapting a vLLM-style cache with page sizes tuned for a different kernel.","solutions":["Use a page_block_size that is a multiple of 32 (e.g. 32, 64, 128) when allocating the KV cache and configuring the builder","If a smaller effective granularity is needed, keep hardware page size 32+ but track partial occupancy via the block table","Verify the cache allocation code applies the same rounded-up page size"],"exampleFix":"// before\nlet page_block_size = 16; // not a multiple of 32\nlet k_cache = Tensor::zeros((n, page_block_size, h_kv, d), ...)?;\n// after\nlet page_block_size = 32; // multiple of 32\nlet k_cache = Tensor::zeros((n, page_block_size, h_kv, d), ...)?;","handlingStrategy":"validation","validationCode":"fn check_page_block_size_multiple(pbs: u32) -> candle_core::Result<()> {\n    if pbs == 0 || pbs % 32 != 0 {\n        candle_core::bail!(\"page_block_size must be a positive multiple of 32, got {pbs}\");\n    }\n    Ok(())\n}\n// call at config load / cache allocation time\ncheck_page_block_size_multiple(page_block_size)?;","typeGuard":"fn is_valid_page_block_size(pbs: u32) -> bool {\n    pbs > 0 && pbs % 32 == 0\n}","tryCatchPattern":"match attn.forward(&q, &k, &v, &sq, &sk, Some(&bt)) {\n    Ok(out) => out,\n    Err(e) if e.to_string().contains(\"multiple of 32\") => {\n        candle_core::bail!(\"invalid page_block_size in cache config: {}\", e)\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Round page sizes up to a multiple of 32 when generating cache configs","Validate page_block_size at config parse time, before any tensor allocation","Document the multiple-of-32 requirement wherever block size is user-tunable"],"tags":["cuda","flash-attention","paged-attention","constraint"],"backgroundTag":"invalid-kernel-constraint","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}