What causes "compile with '--features flash-attn'" in huggingface/candle?

Rebuild with `cargo build --release --features candle-transformers/flash-attn` on CUDA.

compile with '--features flash-attn'

Error message

compile with '--features flash-attn'

What it means

lfm2.rs defines the standard candle `flash_attn` helper whose non-feature stub panics with `unimplemented!`. When the LFM2 model config requests flash attention but the crate lacks the `flash-attn` feature, attention forward reaches the stub and panics.

Source

Thrown at candle-transformers/src/models/lfm2.rs:225

    let on_true = Tensor::new(on_true, on_false.device())?.broadcast_as(shape.dims())?;
    let m = mask.where_cond(&on_true, on_false)?;
    Ok(m)
}

#[cfg(feature = "flash-attn")]
fn flash_attn(
    q: &Tensor,
    k: &Tensor,
    v: &Tensor,
    softmax_scale: f32,
    causal: bool,
) -> Result<Tensor> {
    candle_flash_attn::flash_attn(q, k, v, softmax_scale, causal)
}

#[cfg(not(feature = "flash-attn"))]
fn flash_attn(_: &Tensor, _: &Tensor, _: &Tensor, _: f32, _: bool) -> Result<Tensor> {
    unimplemented!("compile with '--features flash-attn'")
}

/// MLP layer with SwiGLU activation.
#[derive(Debug, Clone)]
struct Mlp {
    gate_proj: Linear,
    up_proj: Linear,
    down_proj: Linear,
    span: tracing::Span,
}

impl Mlp {
    fn new(cfg: &Config, vb: VarBuilder) -> Result<Self> {
        let hidden_size = cfg.hidden_size;
        let intermediate_size = cfg.intermediate_size;
        // LFM2 uses w1 (gate), w3 (up), w2 (down) naming convention
        let gate_proj = linear(hidden_size, intermediate_size, vb.pp("w1"))?;
        let up_proj = linear(hidden_size, intermediate_size, vb.pp("w3"))?;

View on GitHub (pinned to d5fee525bf)

Solutions

Rebuild with `cargo build --release --features candle-transformers/flash-attn` on CUDA.
Set `config.use_flash_attn = false` in code or patch the loaded config.
Use the default attention implementation on unsupported platforms.

Example fix

// before
// built without feature, config.use_flash_attn = true
// after
cargo run --release --features candle-transformers/flash-attn

Defensive patterns

Strategy: validation

Validate before calling

if config.use_flash_attn && !cfg!(feature = "flash-attn") {
    return Err(anyhow::anyhow!("lfm2: flash-attn requires --features candle-transformers/flash-attn (CUDA)"));
}

Type guard

fn flash_attn_enabled_safely(cfg: &lfm2::Config) -> bool {
    cfg.use_flash_attn && cfg!(feature = "flash-attn")
}

Try / catch

if !flash_attn_enabled_safely(&config) {
    config.use_flash_attn = false; // fallback path
}

Prevention

Check feature availability before enabling flash in configs
Keep model configs and cargo features versioned together
Test inference entry points without the feature to catch stub panics early
Use panic hooks or catch_unwind around first forward pass in services

When it happens

Trigger: Running the LFM2 model with `use_flash_attn: true` in Config while the build does not include the `flash-attn` cargo feature.

Common situations: Checkpoint config enables flash attention; builds on CPU/macOS where the CUDA-only backend is unavailable; forgetting `--features` on cargo run/test.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/6bd72ed0fa4bb5e8. Report an issue: GitHub.