paged flash-attn requires page_block_size to be a multiple o

Error message

paged flash-attn requires page_block_size to be a multiple of 32 (got {page_block_size})

What it means

The paged flash-attention CUDA kernel only supports page (block) sizes that are a multiple of 32. After validating the builder value against k's shape, the wrapper checks page_block_size % 32 == 0 and bails otherwise.

Source

Thrown at candle-flash-attn/src/lib.rs:578

            let (_, page_block_size, num_heads_k, k_head_size) = k_l.shape().dims4()?;
            let expected_v = k_l.shape().dims4()?;
            if expected_v != v_l.shape().dims4()? {
                candle::bail!("shape mismatch k {:?} and v {:?}", k_l.shape(), v_l.shape())
            }
            if k_head_size != head_size_og {
                candle::bail!("shape mismatch q {:?} and k {:?}", q_l.shape(), k_l.shape())
            }
            let Some(page_block_size_arg) = self.page_block_size else {
                candle::bail!("paged flash-attn requires page_block_size")
            };
            if page_block_size_arg != page_block_size {
                candle::bail!(
                    "page_block_size {page_block_size_arg} does not match k shape {:?}",
                    k_l.shape()
                )
            }
            if page_block_size % 32 != 0 {
                candle::bail!(
                    "paged flash-attn requires page_block_size to be a multiple of 32 (got {page_block_size})"
                )
            }
            if head_size_og > 512 {
                candle::bail!("paged flash-attn supports head sizes up to 512 (got {head_size_og})")
            }
            (num_heads_k, page_block_size_arg)
        } else {
            let (total_k, num_heads_k, _head_size_og) = k_l.shape().dims3()?;
            let expected_kv = (total_k, num_heads_k, head_size_og);
            if expected_kv != k_l.shape().dims3()? {
                candle::bail!("shape mismatch q {:?} and k {:?}", q_l.shape(), k_l.shape())
            }
            if expected_kv != v_l.shape().dims3()? {
                candle::bail!("shape mismatch q {:?} and v {:?}", q_l.shape(), v_l.shape())
            }
            (num_heads_k, 0)
        };

View on GitHub (pinned to d5fee525bf)

Solutions

Use a page_block_size that is a multiple of 32 (e.g. 32, 64, 128) when allocating the KV cache and configuring the builder
If a smaller effective granularity is needed, keep hardware page size 32+ but track partial occupancy via the block table
Verify the cache allocation code applies the same rounded-up page size

Example fix

// before
let page_block_size = 16; // not a multiple of 32
let k_cache = Tensor::zeros((n, page_block_size, h_kv, d), ...)?;
// after
let page_block_size = 32; // multiple of 32
let k_cache = Tensor::zeros((n, page_block_size, h_kv, d), ...)?;

Defensive patterns

Strategy: validation

Validate before calling

fn check_page_block_size_multiple(pbs: u32) -> candle_core::Result<()> {
    if pbs == 0 || pbs % 32 != 0 {
        candle_core::bail!("page_block_size must be a positive multiple of 32, got {pbs}");
    }
    Ok(())
}
// call at config load / cache allocation time
check_page_block_size_multiple(page_block_size)?;

Type guard

fn is_valid_page_block_size(pbs: u32) -> bool {
    pbs > 0 && pbs % 32 == 0
}

Try / catch

match attn.forward(&q, &k, &v, &sq, &sk, Some(&bt)) {
    Ok(out) => out,
    Err(e) if e.to_string().contains("multiple of 32") => {
        candle_core::bail!("invalid page_block_size in cache config: {}", e)
    }
    Err(e) => return Err(e),
}

Prevention

Round page sizes up to a multiple of 32 when generating cache configs
Validate page_block_size at config parse time, before any tensor allocation
Document the multiple-of-32 requirement wherever block size is user-tunable

When it happens

Trigger: Calling forward in paged mode where the validated page_block_size (from k's shape, which equals the builder value) is e.g. 16, 48, or any non-multiple of 32.

Common situations: Choosing a small page size (e.g. 16) to reduce memory fragmentation in a paged KV cache; adapting a vLLM-style cache with page sizes tuned for a different kernel.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/fedfd9903c238f7b. Report an issue: GitHub.