{"record":{"id":"9ea647375f686372","repo":"huggingface/candle","slug":"compile-with-features-flash-attn-9ea647","errorCode":null,"errorMessage":"compile with '--features flash-attn'","messagePattern":"compile with '--features flash-attn'","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/gemma4/text.rs","lineNumber":207,"sourceCode":"    }\n}\n\n// ── Flash attention ─────────────────────────────────────────────────────────\n\n#[cfg(feature = \"flash-attn\")]\nfn flash_attn(\n    q: &Tensor,\n    k: &Tensor,\n    v: &Tensor,\n    softmax_scale: f32,\n    causal: bool,\n) -> Result<Tensor> {\n    candle_flash_attn::flash_attn(q, k, v, softmax_scale, causal)\n}\n\n#[cfg(not(feature = \"flash-attn\"))]\nfn flash_attn(_: &Tensor, _: &Tensor, _: &Tensor, _: f32, _: bool) -> Result<Tensor> {\n    unimplemented!(\"compile with '--features flash-attn'\")\n}\n\n// ── KvCache ─────────────────────────────────────────────────────────────────\n\n#[derive(Debug, Clone)]\nenum KvCache {\n    Normal(candle_nn::kv_cache::KvCache),\n    Rotating(candle_nn::kv_cache::RotatingKvCache),\n}\n\n// ── Attention ───────────────────────────────────────────────────────────────\n\n#[derive(Debug, Clone)]\nstruct Attention {\n    q_proj: Linear,\n    k_proj: Linear,\n    v_proj: Linear,\n    o_proj: Linear,","sourceCodeStart":189,"sourceCodeEnd":225,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/gemma4/text.rs#L189-L225","documentation":"In gemma4/text.rs the `flash_attn` helper's non-`flash-attn` build stub calls `unimplemented!`. The attention forward pass calls this helper when flash attention is requested, so a missing feature flag turns an unsupported runtime path into a panic.","triggerScenarios":"Running the Gemma4 text model with `use_flash_attn: true` in Config (often inherited from the checkpoint's config.json) while the crate is compiled without the `flash-attn` feature.","commonSituations":"Loading a checkpoint whose config enables flash attention; copying flash-attn example code without enabling the feature; running on CPU/macOS where the CUDA-only feature is unavailable.","solutions":["Build with `cargo build --release --features candle-transformers/flash-attn` on CUDA.","Set `config.use_flash_attn = false` so the model uses the regular softmax attention path.","Regenerate/edit the model config (config.json) to remove flash-attention if you cannot rebuild with the feature."],"exampleFix":"// before\nlet config = Config::from_reader(&mut f)?; // use_flash_attn: true\n// after\nlet mut config = Config::from_reader(&mut f)?;\nconfig.use_flash_attn = false; // or enable the flash-attn cargo feature","handlingStrategy":"validation","validationCode":"if config.use_flash_attn {\n    #[cfg(not(feature = \"flash-attn\"))]\n    panic!(\"gemma4: use_flash_attn requires --features candle-transformers/flash-attn\");\n}","typeGuard":"fn flash_supported(use_flash_attn: bool) -> bool {\n    !use_flash_attn || cfg!(feature = \"flash-attn\")\n}","tryCatchPattern":"if !flash_supported(config.use_flash_attn) {\n    config.use_flash_attn = false; // sanitize before Model::new\n}","preventionTips":["Sanitize loaded configs before constructing models","Never enable flash attention on non-CUDA targets","Pair feature flags and config flags in one place (builder/CLI)","Add a smoke test that runs one forward pass per feature combination"],"tags":["rust","candle","feature-flag","flash-attention","panic"],"backgroundTag":"missing-cargo-feature","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}