huggingface/candle · error

paged flash-attn requires page_block_size

Error message

paged flash-attn requires page_block_size

What it means

Paged varlen flash-attention needs to know the page (block) size both from the block table configuration and to validate k's dims. The wrapper requires the FlashAttentionVarLen struct to have been constructed with page_block_size set (via the builder) whenever paged attention is used; otherwise it bails.

Source

Thrown at candle-flash-attn/src/lib.rs:569

        if k_stride[k_rank - 1] != 1 {
            candle::bail!("the last dim of k must be contiguous {k_stride:?}")
        }
        if v_stride[v_rank - 1] != 1 {
            candle::bail!("the last dim of v must be contiguous {v_stride:?}")
        }

        let (total_q, num_heads, head_size_og) = q_l.shape().dims3()?;
        let (num_heads_k, page_block_size) = if paged {
            let (_, page_block_size, num_heads_k, k_head_size) = k_l.shape().dims4()?;
            let expected_v = k_l.shape().dims4()?;
            if expected_v != v_l.shape().dims4()? {
                candle::bail!("shape mismatch k {:?} and v {:?}", k_l.shape(), v_l.shape())
            }
            if k_head_size != head_size_og {
                candle::bail!("shape mismatch q {:?} and k {:?}", q_l.shape(), k_l.shape())
            }
            let Some(page_block_size_arg) = self.page_block_size else {
                candle::bail!("paged flash-attn requires page_block_size")
            };
            if page_block_size_arg != page_block_size {
                candle::bail!(
                    "page_block_size {page_block_size_arg} does not match k shape {:?}",
                    k_l.shape()
                )
            }
            if page_block_size % 32 != 0 {
                candle::bail!(
                    "paged flash-attn requires page_block_size to be a multiple of 32 (got {page_block_size})"
                )
            }
            if head_size_og > 512 {
                candle::bail!("paged flash-attn supports head sizes up to 512 (got {head_size_og})")
            }
            (num_heads_k, page_block_size_arg)
        } else {
            let (total_k, num_heads_k, _head_size_og) = k_l.shape().dims3()?;

View on GitHub (pinned to d5fee525bf)

Solutions

1. Set page_block_size in the builder: FlashAttentionVarLen::new(...)?.page_block_size(block_size)? (use Some(block_size))
2. Only pass a block_table when the struct is configured with a matching page_block_size
3. Verify the config that constructs the attention struct propagates the paged-cache block size

Example fix

// before
let attn = FlashAttentionVarLen::new(softmax_scale, window_len_left, window_len_right, None)?;
attn.forward(&q, &k, &v, &sq, &sk, Some(&block_table))?;
// after
let attn = FlashAttentionVarLen::new(softmax_scale, window_len_left, window_len_right, Some(block_size))?;
attn.forward(&q, &k, &v, &sq, &sk, Some(&block_table))?;

// Before calling forward in paged mode:
if block_table.is_some() && page_block_size.is_none() {
    return Err(candle_core::Error::msg(
        "block_table supplied but page_block_size was not set on FlashAttentionVarLen",
    ));
}

fn paged_config_ok(page_block_size: &Option<u32>, has_block_table: bool) -> bool {
    !has_block_table || page_block_size.is_some()
}

match attn.forward(&q, &k, &v, &sq, &sk, Some(&bt)) {
    Ok(out) => out,
    Err(e) if e.to_string().contains("requires page_block_size") => {
        candle_core::bail!("builder misconfiguration: set .page_block_size(n) when using a block_table: {}", e)
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling forward with block_table = Some(...) while the FlashAttentionVarLen instance was built without calling .page_block_size(p) in its builder.

Common situations: Enabling paged attention by only adding a block_table argument but forgetting the builder option; constructing attention config generically where the paged branch skips setting page_block_size.

Understand the failure class

Background: "Must pass :limit option" / "Missing required option" — required option errors explained — this error's family across 41 libraries.

Related errors

• block_table must be a cuda tensor
• block_table last dimension must be contiguous
• flash-attn-varlen paged expects k/v tensors of rank 4 (k: {k
• shape mismatch k {:?} and v {:?}
• page_block_size {page_block_size_arg} does not match k shape

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/7df7c0e383cb528a. Report an issue: GitHub.