huggingface/candle · error

compile with '--features flash-attn'

Error message

compile with '--features flash-attn'

What it means

mistral.rs uses the shared candle pattern: a `flash_attn` helper that becomes an `unimplemented!` stub without the `flash-attn` cargo feature. Requesting flash attention in the Mistral model config at runtime hits the stub and panics.

Source

Thrown at candle-transformers/src/models/mistral.rs:199

        let rhs = xs.apply(&self.up_proj)?;
        (lhs * rhs)?.apply(&self.down_proj)
    }
}

#[cfg(feature = "flash-attn")]
fn flash_attn(
    q: &Tensor,
    k: &Tensor,
    v: &Tensor,
    softmax_scale: f32,
    causal: bool,
) -> Result<Tensor> {
    candle_flash_attn::flash_attn(q, k, v, softmax_scale, causal)
}

#[cfg(not(feature = "flash-attn"))]
fn flash_attn(_: &Tensor, _: &Tensor, _: &Tensor, _: f32, _: bool) -> Result<Tensor> {
    unimplemented!("compile with '--features flash-attn'")
}

#[derive(Debug, Clone)]
struct Attention {
    q_proj: Linear,
    k_proj: Linear,
    v_proj: Linear,
    o_proj: Linear,
    num_heads: usize,
    num_kv_heads: usize,
    num_kv_groups: usize,
    head_dim: usize,
    rotary_emb: Arc<RotaryEmbedding>,
    kv_cache: Option<(Tensor, Tensor)>,
    use_flash_attn: bool,
}

impl Attention {

View on GitHub (pinned to d5fee525bf)

Solutions

1. Enable the feature on CUDA: `cargo run --release --features candle-transformers/flash-attn`.
2. Set `config.use_flash_attn = false` before constructing the model.
3. Ensure build scripts/CI consistently pass the feature flag wherever flash attention is expected.

Example fix

// before
config.use_flash_attn = true;
cargo run --release
// after
cargo run --release --features candle-transformers/flash-attn

if config.use_flash_attn && !cfg!(feature = "flash-attn") {
    eprintln!("mistral: flash-attn unavailable; using standard attention");
    config.use_flash_attn = false;
}

fn flash_safe(cfg: &mistral::Config) -> bool {
    !cfg.use_flash_attn || cfg!(feature = "flash-attn")
}

let config = sanitize(config); // where sanitize disables flash unless cfg!(feature = "flash-attn")

When it happens

Trigger: Running a Mistral model with `use_flash_attn: true` in Config (often from the checkpoint's config.json) while built without `--features flash-attn`.

Common situations: Default builds without features; non-CUDA environments; users copying mistral examples that enable flash attention without matching cargo flags.

Related errors

• compile with '--features flash-attn'
• compile with '--features flash-attn'
• compile with '--features flash-attn'
• compile with '--features flash-attn'
• compile with '--features flash-attn'

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/1cae34644d894c1a. Report an issue: GitHub.