What causes "compile with '--features flash-attn'" in huggingface/candle?

Enable the feature: `cargo run --release --features candle-transformers/flash-attn` (CUDA only).

compile with '--features flash-attn'

Error message

compile with '--features flash-attn'

What it means

helium.rs contains the same guarded `flash_attn` helper: without the `flash-attn` cargo feature it is a stub that panics with `unimplemented!`. The Attention forward path calls it whenever flash attention is requested, producing a runtime panic.

Source

Thrown at candle-transformers/src/models/helium.rs:149

        let rhs = xs.apply(&self.up_proj)?;
        (lhs * rhs)?.apply(&self.down_proj)
    }
}

#[cfg(feature = "flash-attn")]
fn flash_attn(
    q: &Tensor,
    k: &Tensor,
    v: &Tensor,
    softmax_scale: f32,
    causal: bool,
) -> Result<Tensor> {
    candle_flash_attn::flash_attn(q, k, v, softmax_scale, causal)
}

#[cfg(not(feature = "flash-attn"))]
fn flash_attn(_: &Tensor, _: &Tensor, _: &Tensor, _: f32, _: bool) -> Result<Tensor> {
    unimplemented!("compile with '--features flash-attn'")
}

#[derive(Debug, Clone)]
struct Attention {
    q_proj: Linear,
    k_proj: Linear,
    v_proj: Linear,
    o_proj: Linear,
    num_heads: usize,
    num_kv_heads: usize,
    num_kv_groups: usize,
    head_dim: usize,
    rotary_emb: Arc<RotaryEmbedding>,
    kv_cache: Option<(Tensor, Tensor)>,
    use_flash_attn: bool,
}

impl Attention {

View on GitHub (pinned to d5fee525bf)

Solutions

Enable the feature: `cargo run --release --features candle-transformers/flash-attn` (CUDA only).
Override `config.use_flash_attn = false` before building the model.
Persist the correct flags in your build script/CI so feature and config always agree.

Example fix

// before
let model = Model::new(&vs, &config)?; // config.use_flash_attn == true, no feature
// after
let mut config = config.clone();
config.use_flash_attn = false;
let model = Model::new(&vs, &config)?;

Defensive patterns

Strategy: validation

Validate before calling

let mut config = helium::Config::from_reader(&mut f)?;
if config.use_flash_attn && !cfg!(feature = "flash-attn") {
    config.use_flash_attn = false;
}

Type guard

fn use_standard_attention(use_flash: bool) -> bool {
    use_flash && !cfg!(feature = "flash-attn")
}

Try / catch

if use_standard_attention(config.use_flash_attn) {
    config.use_flash_attn = false; // prevent unimplemented! panic
}

Prevention

Always normalize checkpoint configs to the built feature set
Add CI jobs for both with/without flash-attn feature builds
Note in CLI help that flash attention needs the cargo feature plus CUDA
Fail fast at startup with a clear message instead of mid-inference panic

When it happens

Trigger: Running the Helium model with `use_flash_attn: true` in Config (e.g. from checkpoint config.json) while compiled without `--features flash-attn`.

Common situations: Default cargo builds without features; non-CUDA environments; copying example code enabling flash attention without matching cargo flags.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/a53bd1904ccf0538. Report an issue: GitHub.