{"record":{"id":"dabb5ae75eb73777","repo":"huggingface/candle","slug":"compile-with-features-flash-attn-dabb5a","errorCode":null,"errorMessage":"compile with '--features flash-attn'","messagePattern":"compile with '--features flash-attn'","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/gemma3.rs","lineNumber":305,"sourceCode":"            KvCache::Rotating(c) => c.reset(),\n        }\n    }\n}\n\n#[cfg(feature = \"flash-attn\")]\nfn flash_attn(\n    q: &Tensor,\n    k: &Tensor,\n    v: &Tensor,\n    softmax_scale: f32,\n    causal: bool,\n) -> Result<Tensor> {\n    candle_flash_attn::flash_attn(q, k, v, softmax_scale, causal)\n}\n\n#[cfg(not(feature = \"flash-attn\"))]\nfn flash_attn(_: &Tensor, _: &Tensor, _: &Tensor, _: f32, _: bool) -> Result<Tensor> {\n    unimplemented!(\"compile with '--features flash-attn'\")\n}\n\n#[derive(Debug, Clone)]\nstruct DecoderLayer {\n    self_attn: Attention,\n    mlp: MLP,\n    input_layernorm: RmsNorm,\n    pre_feedforward_layernorm: RmsNorm,\n    post_feedforward_layernorm: RmsNorm,\n    post_attention_layernorm: RmsNorm,\n    sliding_window: Option<usize>,\n}\n\nimpl DecoderLayer {\n    fn new(\n        use_flash_attn: bool,\n        cfg: &Config,\n        vb: VarBuilder,","sourceCodeStart":287,"sourceCodeEnd":323,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/gemma3.rs#L287-L323","documentation":"Same pattern as other candle models: gemma3.rs contains a `flash_attn` helper whose `#[cfg(not(feature = \"flash-attn\"))]` fallback stubs the call with `unimplemented!`. If the model config requests flash attention but the crate lacks the feature, the stub panics at runtime.","triggerScenarios":"Instantiating/running the Gemma3 text model with `use_flash_attn: true` in Config while compiled without the `flash-attn` cargo feature.","commonSituations":"Checkpoint config.json enables flash attention; default `cargo run` without features; attempting flash attention on a non-CUDA host where the feature cannot be enabled.","solutions":["Rebuild with `--features candle-transformers/flash-attn` on a CUDA-capable machine.","Force `config.use_flash_attn = false` before constructing the model so standard SDPA attention is used.","Verify the target platform supports candle-flash-attn (CUDA only); otherwise stay on the default path."],"exampleFix":"// before\ncargo run --release -- --model gemma3\n// after\ncargo run --release --features candle-transformers/flash-attn -- --model gemma3","handlingStrategy":"validation","validationCode":"let mut config = gemma3::Config::from_json(json)?;\nif config.use_flash_attn && !cfg!(feature = \"flash-attn\") {\n    eprintln!(\"flash-attn feature not enabled; falling back to standard attention\");\n    config.use_flash_attn = false;\n}","typeGuard":"fn can_use_flash_attn(cfg: &gemma3::Config) -> bool {\n    !cfg.use_flash_attn || cfg!(feature = \"flash-attn\")\n}","tryCatchPattern":"std::panic::catch_unwind(|| model.forward(&input)).map_err(|_| anyhow::anyhow!(\"flash_attn unimplemented; rebuild with --features candle-transformers/flash-attn\"))","preventionTips":["Override use_flash_attn=false right after loading checkpoint configs","Use cargo test --features candle-transformers/flash-attn in CI to keep the path compiled","Document required cargo features next to model entry points","Gate flash usage on cfg!(feature = \"flash-attn\") rather than config alone"],"tags":["rust","candle","feature-flag","flash-attention","panic"],"backgroundTag":"missing-cargo-feature","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}