{"record":{"id":"7df7c0e383cb528a","repo":"huggingface/candle","slug":"paged-flash-attn-requires-page-block-size","errorCode":null,"errorMessage":"paged flash-attn requires page_block_size","messagePattern":"paged flash-attn requires page_block_size","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-flash-attn/src/lib.rs","lineNumber":569,"sourceCode":"        if k_stride[k_rank - 1] != 1 {\n            candle::bail!(\"the last dim of k must be contiguous {k_stride:?}\")\n        }\n        if v_stride[v_rank - 1] != 1 {\n            candle::bail!(\"the last dim of v must be contiguous {v_stride:?}\")\n        }\n\n        let (total_q, num_heads, head_size_og) = q_l.shape().dims3()?;\n        let (num_heads_k, page_block_size) = if paged {\n            let (_, page_block_size, num_heads_k, k_head_size) = k_l.shape().dims4()?;\n            let expected_v = k_l.shape().dims4()?;\n            if expected_v != v_l.shape().dims4()? {\n                candle::bail!(\"shape mismatch k {:?} and v {:?}\", k_l.shape(), v_l.shape())\n            }\n            if k_head_size != head_size_og {\n                candle::bail!(\"shape mismatch q {:?} and k {:?}\", q_l.shape(), k_l.shape())\n            }\n            let Some(page_block_size_arg) = self.page_block_size else {\n                candle::bail!(\"paged flash-attn requires page_block_size\")\n            };\n            if page_block_size_arg != page_block_size {\n                candle::bail!(\n                    \"page_block_size {page_block_size_arg} does not match k shape {:?}\",\n                    k_l.shape()\n                )\n            }\n            if page_block_size % 32 != 0 {\n                candle::bail!(\n                    \"paged flash-attn requires page_block_size to be a multiple of 32 (got {page_block_size})\"\n                )\n            }\n            if head_size_og > 512 {\n                candle::bail!(\"paged flash-attn supports head sizes up to 512 (got {head_size_og})\")\n            }\n            (num_heads_k, page_block_size_arg)\n        } else {\n            let (total_k, num_heads_k, _head_size_og) = k_l.shape().dims3()?;","sourceCodeStart":551,"sourceCodeEnd":587,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-flash-attn/src/lib.rs#L551-L587","documentation":"Paged varlen flash-attention needs to know the page (block) size both from the block table configuration and to validate k's dims. The wrapper requires the FlashAttentionVarLen struct to have been constructed with page_block_size set (via the builder) whenever paged attention is used; otherwise it bails.","triggerScenarios":"Calling forward with block_table = Some(...) while the FlashAttentionVarLen instance was built without calling .page_block_size(p) in its builder.","commonSituations":"Enabling paged attention by only adding a block_table argument but forgetting the builder option; constructing attention config generically where the paged branch skips setting page_block_size.","solutions":["Set page_block_size in the builder: FlashAttentionVarLen::new(...)?.page_block_size(block_size)? (use Some(block_size))","Only pass a block_table when the struct is configured with a matching page_block_size","Verify the config that constructs the attention struct propagates the paged-cache block size"],"exampleFix":"// before\nlet attn = FlashAttentionVarLen::new(softmax_scale, window_len_left, window_len_right, None)?;\nattn.forward(&q, &k, &v, &sq, &sk, Some(&block_table))?;\n// after\nlet attn = FlashAttentionVarLen::new(softmax_scale, window_len_left, window_len_right, Some(block_size))?;\nattn.forward(&q, &k, &v, &sq, &sk, Some(&block_table))?;","handlingStrategy":"validation","validationCode":"// Before calling forward in paged mode:\nif block_table.is_some() && page_block_size.is_none() {\n    return Err(candle_core::Error::msg(\n        \"block_table supplied but page_block_size was not set on FlashAttentionVarLen\",\n    ));\n}","typeGuard":"fn paged_config_ok(page_block_size: &Option<u32>, has_block_table: bool) -> bool {\n    !has_block_table || page_block_size.is_some()\n}","tryCatchPattern":"match attn.forward(&q, &k, &v, &sq, &sk, Some(&bt)) {\n    Ok(out) => out,\n    Err(e) if e.to_string().contains(\"requires page_block_size\") => {\n        candle_core::bail!(\"builder misconfiguration: set .page_block_size(n) when using a block_table: {}\", e)\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Always configure page_block_size in the same code path that enables paged attention","Encapsulate paged-attention setup in a constructor that takes the block size","Fail fast at config load if block_table paging is enabled but block size is absent"],"tags":["cuda","flash-attention","paged-attention","missing-config"],"backgroundTag":"missing-required-option","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}