{"record":{"id":"62417b2ec8f8890c","repo":"huggingface/candle","slug":"only-multi-query-attention-true-is-supported-62417b","errorCode":null,"errorMessage":"only multi_query_attention=true is supported","messagePattern":"only multi_query_attention=true is supported","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/glm4.rs","lineNumber":286,"sourceCode":"            num_multi_query_groups_per_partition: cfg.multi_query_group_num,\n            hidden_size_per_attention_head: cfg.kv_channels,\n            kv_cache: None,\n        })\n    }\n\n    fn reset_kv_cache(&mut self) {\n        self.kv_cache = None\n    }\n\n    fn forward(\n        &mut self,\n        xs: &Tensor,\n        attention_mask: &Option<Tensor>,\n        rotary_emb: &RotaryEmbedding,\n    ) -> Result<Tensor> {\n        let mixed_x_layer = xs.apply(&self.query_key_value)?;\n        if !self.multi_query_attention {\n            candle::bail!(\"only multi_query_attention=true is supported\")\n        }\n        let hpa = self.hidden_size_per_attention_head;\n        let query_layer =\n            mixed_x_layer.narrow(D::Minus1, 0, self.num_attention_heads_per_partition * hpa)?;\n        let key_layer = mixed_x_layer.narrow(\n            D::Minus1,\n            self.num_attention_heads_per_partition * hpa,\n            self.num_multi_query_groups_per_partition * hpa,\n        )?;\n        let value_layer = mixed_x_layer.narrow(\n            D::Minus1,\n            self.num_attention_heads_per_partition * hpa\n                + self.num_multi_query_groups_per_partition * hpa,\n            self.num_multi_query_groups_per_partition * hpa,\n        )?;\n        let query_layer = query_layer.reshape((\n            query_layer.dim(0)?,\n            query_layer.dim(1)?,","sourceCodeStart":268,"sourceCodeEnd":304,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/glm4.rs#L268-L304","documentation":"GLM-4 attention in candle only implements multi-query attention (MQA/GQA) where key/value heads are fewer than query heads. Attention::forward checks the config flag multi_query_attention and bails if the checkpoint uses standard multi-head attention, since the narrow-based QKV splitting assumes that layout.","triggerScenarios":"Loading a GLM-4 checkpoint whose config has multi_query_attention=false (or missing, defaulting to false) and running a forward pass.","commonSituations":"Using a base GLM variant/config that doesn't set multi_query_attention=true; a truncated or hand-made config.json missing the key; adapting code from another GLM size that used MHA.","solutions":["Set \"multi_query_attention\": true in the model config before loading","Verify you downloaded the intended GLM-4 checkpoint variant that uses MQA/GQA","Extend the candle implementation if you genuinely need MHA support"],"exampleFix":"// before (config.json)\n{\"multi_query_attention\": false, ...}\n// after\n{\"multi_query_attention\": true, ...}","handlingStrategy":"validation","validationCode":"if cfg.multi_query_attention != true {\n    return Err(\"this checkpoint is not multi-query attention; candle glm4 only supports MQA\");\n}","typeGuard":"fn is_mqa(c: &glm4::Config) -> bool { c.multi_query_attention }","tryCatchPattern":"match model.forward(&xs, &positions, &mask) {\n    Err(e) if e.to_string().contains(\"only multi_query_attention\") =>\n        Err(anyhow!(\"set multi_query_attention=true or use a different model impl\")),\n    r => r.map_err(Into::into),\n}","preventionTips":["Check multi_query_attention in config.json before loading GLM-4 checkpoints","Download the intended GLM-4 variant known to use MQA/GQA","Don't reuse configs from other GLM sizes without review"],"tags":["config","glm4","unsupported-feature"],"backgroundTag":"unsupported-architecture-config","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}