huggingface/candle · error

flash_attn_varlen_cpu is CPU only

Error message

flash_attn_varlen_cpu is CPU only

What it means

Error "flash_attn_varlen_cpu is CPU only" thrown in huggingface/candle.

Source

Thrown at candle-nn/src/attention/cpu_flash/varlen.rs:42

///
/// **Output shape:** `[total_q, Hq, D]`
#[allow(clippy::too_many_arguments)]
pub fn flash_attn_varlen_cpu(
    q: &Tensor,                    // [total_q, Hq, D]
    k: &Tensor,                    // [total_k, Hk, D]
    v: &Tensor,                    // [total_k, Hv, D]
    alibi_slopes: Option<&Tensor>, // [Hq]
    seqlens_q: &Tensor,            // [B]
    seqlens_k: &Tensor,            // [B]
    _max_seqlen_q: usize,
    _max_seqlen_k: usize,
    softmax_scale: f32,
    causal: bool,
    window_left: Option<usize>,
    window_right: Option<usize>,
) -> Result<Tensor> {
    if !q.device().is_cpu() {
        candle::bail!("flash_attn_varlen_cpu is CPU only");
    }

    let dt = q.dtype();
    if k.dtype() != dt || v.dtype() != dt {
        candle::bail!(
            "q/k/v must have the same dtype (got q={:?}, k={:?}, v={:?})",
            dt,
            k.dtype(),
            v.dtype()
        );
    }
    if dt != DType::F32 && dt != DType::F16 {
        candle::bail!("flash_attn_varlen_cpu supports f32 or f16 only");
    }

    let (total_q, hq, d) = q.dims3()?;
    let (total_k, hk, dk) = k.dims3()?;
    let (_total_v, hv, dv) = v.dims3()?;

View on GitHub (pinned to d5fee525bf)

When it happens

Trigger: Thrown at candle-nn/src/attention/cpu_flash/varlen.rs:42 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/592d26c75db148b7. Report an issue: GitHub.