{"record":{"id":"223b317e3865e6b2","repo":"huggingface/candle","slug":"n-head-kv-is-not-supported","errorCode":null,"errorMessage":"n_head_kv is not supported","messagePattern":"n_head_kv is not supported","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/falcon.rs","lineNumber":82,"sourceCode":"            alibi: false,\n            new_decoder_architecture: false,\n            multi_query: true,\n            parallel_attn: true,\n            bias: false,\n        }\n    }\n}\n\nimpl Config {\n    pub fn validate(&self) -> Result<()> {\n        if self.alibi {\n            candle::bail!(\"alibi is not supported\");\n        }\n        if self.new_decoder_architecture {\n            candle::bail!(\"new_decoder_architecture is not supported\");\n        }\n        if self.n_head_kv.is_some() {\n            candle::bail!(\"n_head_kv is not supported\");\n        }\n        Ok(())\n    }\n\n    // https://huggingface.co/tiiuae/falcon-7b/blob/main/config.json\n    pub fn falcon7b() -> Self {\n        // This is currently on par with the defaults, the defaults come from the Python default\n        // arguments for the config initialization whereas the following come from the json config.\n        Self {\n            vocab_size: 65024,\n            hidden_size: 4544,\n            num_hidden_layers: 32,\n            num_attention_heads: 71,\n            layer_norm_epsilon: 1e-5,\n            initializer_range: 0.02,\n            use_cache: true,\n            bos_token_id: 11,\n            eos_token_id: 11,","sourceCodeStart":64,"sourceCodeEnd":100,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/falcon.rs#L64-L100","documentation":"Falcon's Config::validate rejects configs where n_head_kv is Some. n_head_kv (grouped-query / multi-query attention) belongs to newer Falcon variants that candle's Falcon implementation does not support, so the config is refused up front.","triggerScenarios":"Deserializing a Falcon config.json that contains a non-null \"n_head_kv\" (multi-query checkpoints like falcon-40b) and calling validate() before building the model.","commonSituations":"Selecting a Falcon checkpoint with grouped-query attention from HuggingFace; candle's supported falcon-7b config has n_head_kv: None.","solutions":["Use a checkpoint without grouped-query attention (config.json has no n_head_kv or null), e.g. tiiuae/falcon-7b.","Use Config::falcon7b() or another built-in preset that already passes validate().","If MQA is required, use a different candle model implementation that supports it or extend falcon.rs."],"exampleFix":"// before: falcon-40b config.json with \"n_head_kv\": 8 -> validate() bails\n// after: use a supported preset\nlet config = Config::falcon7b();\nconfig.validate()?;","handlingStrategy":"validation","validationCode":"if config.n_head_kv.is_some() {\n    return Err(anyhow::anyhow!(\"checkpoint uses grouped-query attention (n_head_kv); not supported by candle falcon\"));\n}\nconfig.validate()?;","typeGuard":"fn supports_no_mqa(c: &Config) -> bool { c.n_head_kv.is_none() }","tryCatchPattern":"if let Err(e) = config.validate() {\n    if e.to_string().contains(\"n_head_kv\") {\n        anyhow::bail!(\"use a falcon checkpoint without grouped-query attention (e.g. falcon-7b)\");\n    }\n    return Err(e.into());\n}","preventionTips":["Check config.json for n_head_kv before downloading a falcon checkpoint.","Use falcon-7b-family checkpoints, which have no n_head_kv.","Route MQA/GQA requirements to a candle model that implements them."],"tags":["config","falcon","grouped-query-attention","candle"],"backgroundTag":"unsupported-model-config","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}