{"record":{"id":"ca1b9495f7aade36","repo":"huggingface/candle","slug":"compile-with-features-flash-attn-ca1b94","errorCode":null,"errorMessage":"compile with '--features flash-attn'","messagePattern":"compile with '--features flash-attn'","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/gemma2.rs","lineNumber":279,"sourceCode":"    fn clear_kv_cache(&mut self) {\n        self.kv_cache = None\n    }\n}\n\n#[cfg(feature = \"flash-attn\")]\nfn flash_attn(\n    q: &Tensor,\n    k: &Tensor,\n    v: &Tensor,\n    softmax_scale: f32,\n    causal: bool,\n) -> Result<Tensor> {\n    candle_flash_attn::flash_attn(q, k, v, softmax_scale, causal)\n}\n\n#[cfg(not(feature = \"flash-attn\"))]\nfn flash_attn(_: &Tensor, _: &Tensor, _: &Tensor, _: f32, _: bool) -> Result<Tensor> {\n    unimplemented!(\"compile with '--features flash-attn'\")\n}\n\n#[derive(Debug, Clone)]\nstruct DecoderLayer {\n    self_attn: Attention,\n    mlp: MLP,\n    input_layernorm: RmsNorm,\n    pre_feedforward_layernorm: RmsNorm,\n    post_feedforward_layernorm: RmsNorm,\n    post_attention_layernorm: RmsNorm,\n}\n\nimpl DecoderLayer {\n    fn new(\n        rotary_emb: Arc<RotaryEmbedding>,\n        use_flash_attn: bool,\n        cfg: &Config,\n        vb: VarBuilder,","sourceCodeStart":261,"sourceCodeEnd":297,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/gemma2.rs#L261-L297","documentation":"In candle-transformers, each model defines a `flash_attn` helper. When the crate is compiled without the `flash-attn` cargo feature, a `#[cfg(not(feature = \"flash-attn\"))]` stub is used that immediately panics via `unimplemented!`. The model code still calls `flash_attn` when flash attention is enabled at runtime, so the panic fires instead of computing attention.","triggerScenarios":"Running a Gemma2 model with `use_flash_attn: true` in the model Config (e.g. loaded from a checkpoint whose config.json has flash-attention enabled) while the crate was built without `--features flash-attn`.","commonSituations":"Users load a Hugging Face checkpoint whose config enables flash attention, or copy example code that sets `config.use_flash_attn = true`, but run the default cargo build where the `flash-attn` feature is off (it is also unavailable on non-CUDA platforms).","solutions":["Rebuild with the feature enabled: `cargo build --release --features candle-transformers/flash-attn` (requires a CUDA-capable environment).","Set `config.use_flash_attn = false` in code or edit the loaded config so the non-flash attention path is taken instead.","Run on a CUDA machine; the `candle-flash-attn` backend only supports CUDA, so on CPU/macOS the feature cannot be used."],"exampleFix":"// before\nlet mut config = Config::from_json(json)?;\nconfig.use_flash_attn = true;\n// after\nlet mut config = Config::from_json(json)?;\n// build with: cargo run --release --features candle-transformers/flash-attn\nconfig.use_flash_attn = std::env::var(\"USE_FLASH_ATTN\").is_ok();","handlingStrategy":"validation","validationCode":"#[cfg(not(feature = \"flash-attn\"))]\nfn assert_flash_attn_unsupported() {\n    assert!(!config.use_flash_attn, \"use_flash_attn=true requires building with --features candle-transformers/flash-attn (CUDA only)\");\n}","typeGuard":"fn flash_attn_available(cfg: &gemma2::Config) -> bool {\n    !cfg.use_flash_attn || cfg!(feature = \"flash-attn\")\n}","tryCatchPattern":"// Rust panics here, not Result errors; guard before constructing:\nif config.use_flash_attn && !cfg!(feature = \"flash-attn\") {\n    config.use_flash_attn = false;\n}","preventionTips":["Always keep config.use_flash_attn in sync with the cargo feature set","Compile-time check: add a compile_error! or test asserting the feature when the flag is used","Remember candle-flash-attn is CUDA-only; disable the flag on CPU/macOS builds","Centralize config loading and force-disable flash attention unless explicitly supported"],"tags":["rust","candle","feature-flag","flash-attention","panic"],"backgroundTag":"missing-cargo-feature","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}