What causes "only kv-repeat = 1 is supported" in huggingface/candle?

Set kv_repeat to 1 in the transformer Config

only kv-repeat = 1 is supported

Error message

only kv-repeat = 1 is supported

What it means

StreamingTransformer::forward requires kv_repeat == 1; the mimi attention implementation only supports writing one new KV frame per step. When the config sets a kv-repeat > 1 (used for speculative/chunked KV reordering in the original implementation), the forward pass bails.

Source

Thrown at candle-transformers/src/models/mimi/transformer.rs:151

            k_proj,
            v_proj,
            out_proj,
            rope: rope.clone(),
            kv_repeat: cfg.kv_repeat,
            num_heads: cfg.num_heads,
            context: cfg.context,
            neg_inf,
            kv_cache: candle_nn::kv_cache::RotatingKvCache::new(2, cfg.context),
            pos: 0,
            use_flash_attn: false,
            span: tracing::span!(tracing::Level::TRACE, "mha"),
        })
    }

    pub fn forward(&mut self, xs: &Tensor, mask: Option<&Tensor>) -> Result<Tensor> {
        let _enter = self.span.enter();
        if self.kv_repeat != 1 {
            candle::bail!("only kv-repeat = 1 is supported")
        }
        let (b, t, hd) = xs.dims3()?;
        let head_dim = hd / self.num_heads;
        let q = xs
            .apply(&self.q_proj)?
            .reshape((b, t, self.num_heads, head_dim))?;
        let k = xs
            .apply(&self.k_proj)?
            .reshape((b, t, self.num_heads, head_dim))?;
        let v = xs
            .apply(&self.v_proj)?
            .reshape((b, t, self.num_heads, head_dim))?;
        // qk_layer_norm = None
        // kv_repeat = 1, otherwise we would need repeat_kv
        let mut q = q.transpose(1, 2)?.contiguous()?; // b,h,t,d
        let mut k = k.transpose(1, 2)?.contiguous()?; // b,h,k,d
        let v = v.transpose(1, 2)?.contiguous()?; // b,h,k,d
        if let Some(rope) = &self.rope {

View on GitHub (pinned to d5fee525bf)

Solutions

Set kv_repeat to 1 in the transformer Config
If you need kv_repeat > 1, extend candle's mimi attention to implement the repeat logic

Example fix

// before
let cfg = Config { kv_repeat: 2, ..base };
t.forward(&xs, mask)?; // bails
// after
let cfg = Config { kv_repeat: 1, ..base };
t.forward(&xs, mask)?;

Defensive patterns

Strategy: validation

Validate before calling

if cfg.kv_repeat != 1 {
    return Err(anyhow::anyhow!("candle mimi requires kv_repeat == 1"));
}
let mut t = StreamingTransformer::new(&cfg, vb)?;

Type guard

fn supports_kv_repeat(cfg: &Config) -> bool { cfg.kv_repeat == 1 }

Try / catch

let out = match t.forward(&xs, mask) {
    Ok(o) => o,
    Err(e) if e.to_string().contains("only kv-repeat = 1") => {
        return Err(anyhow::anyhow!("set kv_repeat = 1 in the transformer config"))
    }
    Err(e) => return Err(e.into()),
};

Prevention

Normalize imported mimi configs to kv_repeat = 1
Assert kv_repeat == 1 in a config sanity check at load time
Avoid copying kv_repeat from reference Moshi code defaults

When it happens

Trigger: Calling StreamingTransformer::forward when Config.kv_repeat is anything other than 1, e.g. kv_repeat = 2 loaded from a mimi config that uses delayed/streaming KV replication.

Common situations: Using a mimi config with kv_repeat set for speculative decoding; copying kv_repeat from the reference Moshi implementation.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/0cc65f028ba88010. Report an issue: GitHub.