{"record":{"id":"cadf00cc7d9be7af","repo":"huggingface/candle","slug":"compile-with-features-flash-attn-cadf00","errorCode":null,"errorMessage":"compile with '--features flash-attn'","messagePattern":"compile with '--features flash-attn'","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/llama.rs","lineNumber":258,"sourceCode":"    span: tracing::Span,\n    span_rot: tracing::Span,\n    max_position_embeddings: usize,\n}\n\n#[cfg(feature = \"flash-attn\")]\nfn flash_attn(\n    q: &Tensor,\n    k: &Tensor,\n    v: &Tensor,\n    softmax_scale: f32,\n    causal: bool,\n) -> Result<Tensor> {\n    candle_flash_attn::flash_attn(q, k, v, softmax_scale, causal)\n}\n\n#[cfg(not(feature = \"flash-attn\"))]\nfn flash_attn(_: &Tensor, _: &Tensor, _: &Tensor, _: f32, _: bool) -> Result<Tensor> {\n    unimplemented!(\"compile with '--features flash-attn'\")\n}\n\nimpl CausalSelfAttention {\n    fn apply_rotary_emb(&self, x: &Tensor, index_pos: usize, cache: &Cache) -> Result<Tensor> {\n        let _enter = self.span_rot.enter();\n        let (_b_sz, _, seq_len, _hidden_size) = x.dims4()?;\n        let cos = cache.cos.narrow(0, index_pos, seq_len)?;\n        let sin = cache.sin.narrow(0, index_pos, seq_len)?;\n        candle_nn::rotary_emb::rope(x, &cos, &sin)\n    }\n\n    fn forward(\n        &self,\n        x: &Tensor,\n        index_pos: usize,\n        block_idx: usize,\n        cache: &mut Cache,\n    ) -> Result<Tensor> {","sourceCodeStart":240,"sourceCodeEnd":276,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/llama.rs#L240-L276","documentation":"llama.rs's `flash_attn` helper is replaced by an `unimplemented!` stub when compiled without the `flash-attn` cargo feature. If flash attention is requested in the config, `CausalSelfAttention::forward` calls the stub and the run panics.","triggerScenarios":"Running a LLaMA model with `use_flash_attn: true` in Config (common when reusing llama3/3.1 checkpoint configs) without `--features flash-attn`.","commonSituations":"Checkpoint config.json enables flash attention; default builds; non-CUDA platforms (macOS, CPU-only Linux) where candle-flash-attn cannot be enabled.","solutions":["Build with the feature: `cargo run --release --features candle-transformers/flash-attn` on a CUDA machine.","Set `config.use_flash_attn = false` before instantiating the model.","If porting example code, remove the flash-attn flag unless your build and hardware support it."],"exampleFix":"// before\nlet mut config = llama::Config::from_json(json)?;\nconfig.use_flash_attn = true;\n// after\nlet mut config = llama::Config::from_json(json)?;\nconfig.use_flash_attn = false; // or build with --features candle-transformers/flash-attn","handlingStrategy":"validation","validationCode":"let mut config = llama::Config::from_json(json)?;\nif config.use_flash_attn && !cfg!(feature = \"flash-attn\") {\n    config.use_flash_attn = false;\n}","typeGuard":"fn flash_attention_ok(cfg: &llama::Config) -> bool {\n    !cfg.use_flash_attn || cfg!(feature = \"flash-attn\")\n}","tryCatchPattern":"assert!(flash_attention_ok(&config), \"rebuild with --features candle-transformers/flash-attn or set use_flash_attn=false\");","preventionTips":["Llama 3 configs may enable flash attention; always inspect after loading","Remember the feature needs CUDA; disable on CPU/macOS","Centralize config sanitization for all models in the binary","Add a unit test asserting Model::new succeeds without the feature"],"tags":["rust","candle","feature-flag","flash-attention","panic"],"backgroundTag":"missing-cargo-feature","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}