What causes "compile with '--features flash-attn'" in huggingface/candle?

Rebuild with `--features candle-transformers/flash-attn` on a CUDA machine.

compile with '--features flash-attn'

Error message

compile with '--features flash-attn'

What it means

granitemoehybrid.rs follows the shared candle pattern: a `flash_attn` helper replaced by an `unimplemented!` stub when the `flash-attn` feature is off. Requesting flash attention in the model config at runtime therefore panics instead of computing attention.

Source

Thrown at candle-transformers/src/models/granitemoehybrid.rs:256

    span_rot: tracing::Span,
    max_position_embeddings: usize,
    attention_multiplier: f32,
}

#[cfg(feature = "flash-attn")]
fn flash_attn(
    q: &Tensor,
    k: &Tensor,
    v: &Tensor,
    softmax_scale: f32,
    causal: bool,
) -> Result<Tensor> {
    candle_flash_attn::flash_attn(q, k, v, softmax_scale, causal)
}

#[cfg(not(feature = "flash-attn"))]
fn flash_attn(_: &Tensor, _: &Tensor, _: &Tensor, _: f32, _: bool) -> Result<Tensor> {
    unimplemented!("compile with '--features flash-attn'")
}

impl CausalSelfAttention {
    fn apply_rotary_emb(
        &self,
        x: &Tensor,
        index_pos: usize,
        cache: &GraniteMoeHybridCache,
    ) -> Result<Tensor> {
        let _enter = self.span_rot.enter();
        let (_b_sz, _, seq_len, _hidden_size) = x.dims4()?;
        let cos = cache.cos.narrow(0, index_pos, seq_len)?;
        let sin = cache.sin.narrow(0, index_pos, seq_len)?;
        candle_nn::rotary_emb::rope(x, &cos, &sin)
    }

    fn forward(
        &self,

View on GitHub (pinned to d5fee525bf)

Solutions

Rebuild with `--features candle-transformers/flash-attn` on a CUDA machine.
Set `config.use_flash_attn = false` to use the standard attention path.
Check your platform: on non-CUDA targets the flash-attn feature cannot be enabled, so always disable it in config.

Example fix

// before
config.use_flash_attn = true;
cargo run --release
// after
config.use_flash_attn = false;
cargo run --release

Defensive patterns

Strategy: validation

Validate before calling

if config.use_flash_attn && !cfg!(feature = "flash-attn") {
    config.use_flash_attn = false;
    log::warn!("flash-attn feature missing; disabled at runtime");
}

Type guard

fn flash_attn_usable(cfg: &granitemoehybrid::Config) -> bool {
    !cfg.use_flash_attn || cfg!(feature = "flash-attn")
}

Try / catch

assert!(flash_attn_usable(&config), "enable --features candle-transformers/flash-attn or disable use_flash_attn");

Prevention

Enforce feature-flag/config-flag consistency in a shared constructor
Record required cargo features in README/model docs
Avoid enabling flash attention on macOS/CPU builds
Use cfg!(feature = ...) checks instead of assuming feature availability

When it happens

Trigger: Running the Granite MoE Hybrid model with `use_flash_attn: true` in Config while the crate is built without the `flash-attn` cargo feature.

Common situations: Config from a checkpoint enables flash attention; users on CPU/macOS or Windows (no CUDA support in candle-flash-attn) hit the stub; forgetting the feature flag in cargo build/run commands.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/dccde7beab62d2f2. Report an issue: GitHub.