{"record":{"id":"0cc65f028ba88010","repo":"huggingface/candle","slug":"only-kv-repeat-1-is-supported","errorCode":null,"errorMessage":"only kv-repeat = 1 is supported","messagePattern":"only kv-repeat = 1 is supported","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/mimi/transformer.rs","lineNumber":151,"sourceCode":"            k_proj,\n            v_proj,\n            out_proj,\n            rope: rope.clone(),\n            kv_repeat: cfg.kv_repeat,\n            num_heads: cfg.num_heads,\n            context: cfg.context,\n            neg_inf,\n            kv_cache: candle_nn::kv_cache::RotatingKvCache::new(2, cfg.context),\n            pos: 0,\n            use_flash_attn: false,\n            span: tracing::span!(tracing::Level::TRACE, \"mha\"),\n        })\n    }\n\n    pub fn forward(&mut self, xs: &Tensor, mask: Option<&Tensor>) -> Result<Tensor> {\n        let _enter = self.span.enter();\n        if self.kv_repeat != 1 {\n            candle::bail!(\"only kv-repeat = 1 is supported\")\n        }\n        let (b, t, hd) = xs.dims3()?;\n        let head_dim = hd / self.num_heads;\n        let q = xs\n            .apply(&self.q_proj)?\n            .reshape((b, t, self.num_heads, head_dim))?;\n        let k = xs\n            .apply(&self.k_proj)?\n            .reshape((b, t, self.num_heads, head_dim))?;\n        let v = xs\n            .apply(&self.v_proj)?\n            .reshape((b, t, self.num_heads, head_dim))?;\n        // qk_layer_norm = None\n        // kv_repeat = 1, otherwise we would need repeat_kv\n        let mut q = q.transpose(1, 2)?.contiguous()?; // b,h,t,d\n        let mut k = k.transpose(1, 2)?.contiguous()?; // b,h,k,d\n        let v = v.transpose(1, 2)?.contiguous()?; // b,h,k,d\n        if let Some(rope) = &self.rope {","sourceCodeStart":133,"sourceCodeEnd":169,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/mimi/transformer.rs#L133-L169","documentation":"StreamingTransformer::forward requires kv_repeat == 1; the mimi attention implementation only supports writing one new KV frame per step. When the config sets a kv-repeat > 1 (used for speculative/chunked KV reordering in the original implementation), the forward pass bails.","triggerScenarios":"Calling StreamingTransformer::forward when Config.kv_repeat is anything other than 1, e.g. kv_repeat = 2 loaded from a mimi config that uses delayed/streaming KV replication.","commonSituations":"Using a mimi config with kv_repeat set for speculative decoding; copying kv_repeat from the reference Moshi implementation.","solutions":["Set kv_repeat to 1 in the transformer Config","If you need kv_repeat > 1, extend candle's mimi attention to implement the repeat logic"],"exampleFix":"// before\nlet cfg = Config { kv_repeat: 2, ..base };\nt.forward(&xs, mask)?; // bails\n// after\nlet cfg = Config { kv_repeat: 1, ..base };\nt.forward(&xs, mask)?;","handlingStrategy":"validation","validationCode":"if cfg.kv_repeat != 1 {\n    return Err(anyhow::anyhow!(\"candle mimi requires kv_repeat == 1\"));\n}\nlet mut t = StreamingTransformer::new(&cfg, vb)?;","typeGuard":"fn supports_kv_repeat(cfg: &Config) -> bool { cfg.kv_repeat == 1 }","tryCatchPattern":"let out = match t.forward(&xs, mask) {\n    Ok(o) => o,\n    Err(e) if e.to_string().contains(\"only kv-repeat = 1\") => {\n        return Err(anyhow::anyhow!(\"set kv_repeat = 1 in the transformer config\"))\n    }\n    Err(e) => return Err(e.into()),\n};","preventionTips":["Normalize imported mimi configs to kv_repeat = 1","Assert kv_repeat == 1 in a config sanity check at load time","Avoid copying kv_repeat from reference Moshi code defaults"],"tags":["rust","candle","mimi","transformer","kv-cache"],"backgroundTag":"unsupported-model-config","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}