What causes "compile with '--features flash-attn'" in huggingface/candle?

Build with `cargo build --release --features candle-transformers/flash-attn` on CUDA.

compile with '--features flash-attn'

Error message

compile with '--features flash-attn'

What it means

In gemma4/text.rs the `flash_attn` helper's non-`flash-attn` build stub calls `unimplemented!`. The attention forward pass calls this helper when flash attention is requested, so a missing feature flag turns an unsupported runtime path into a panic.

Source

Thrown at candle-transformers/src/models/gemma4/text.rs:207

    }
}

// ── Flash attention ─────────────────────────────────────────────────────────

#[cfg(feature = "flash-attn")]
fn flash_attn(
    q: &Tensor,
    k: &Tensor,
    v: &Tensor,
    softmax_scale: f32,
    causal: bool,
) -> Result<Tensor> {
    candle_flash_attn::flash_attn(q, k, v, softmax_scale, causal)
}

#[cfg(not(feature = "flash-attn"))]
fn flash_attn(_: &Tensor, _: &Tensor, _: &Tensor, _: f32, _: bool) -> Result<Tensor> {
    unimplemented!("compile with '--features flash-attn'")
}

// ── KvCache ─────────────────────────────────────────────────────────────────

#[derive(Debug, Clone)]
enum KvCache {
    Normal(candle_nn::kv_cache::KvCache),
    Rotating(candle_nn::kv_cache::RotatingKvCache),
}

// ── Attention ───────────────────────────────────────────────────────────────

#[derive(Debug, Clone)]
struct Attention {
    q_proj: Linear,
    k_proj: Linear,
    v_proj: Linear,
    o_proj: Linear,

View on GitHub (pinned to d5fee525bf)

Solutions

Build with `cargo build --release --features candle-transformers/flash-attn` on CUDA.
Set `config.use_flash_attn = false` so the model uses the regular softmax attention path.
Regenerate/edit the model config (config.json) to remove flash-attention if you cannot rebuild with the feature.

Example fix

// before
let config = Config::from_reader(&mut f)?; // use_flash_attn: true
// after
let mut config = Config::from_reader(&mut f)?;
config.use_flash_attn = false; // or enable the flash-attn cargo feature

Defensive patterns

Strategy: validation

Validate before calling

if config.use_flash_attn {
    #[cfg(not(feature = "flash-attn"))]
    panic!("gemma4: use_flash_attn requires --features candle-transformers/flash-attn");
}

Type guard

fn flash_supported(use_flash_attn: bool) -> bool {
    !use_flash_attn || cfg!(feature = "flash-attn")
}

Try / catch

if !flash_supported(config.use_flash_attn) {
    config.use_flash_attn = false; // sanitize before Model::new
}

Prevention

Sanitize loaded configs before constructing models
Never enable flash attention on non-CUDA targets
Pair feature flags and config flags in one place (builder/CLI)
Add a smoke test that runs one forward pass per feature combination

When it happens

Trigger: Running the Gemma4 text model with `use_flash_attn: true` in Config (often inherited from the checkpoint's config.json) while the crate is compiled without the `flash-attn` feature.

Common situations: Loading a checkpoint whose config enables flash attention; copying flash-attn example code without enabling the feature; running on CPU/macOS where the CUDA-only feature is unavailable.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/9ea647375f686372. Report an issue: GitHub.