What causes "kv_cache_enabled=true is not supported" in huggingface/candle?

Set kv_cache_enabled to false in the Config before constructing the model

kv_cache_enabled=true is not supported

Error message

kv_cache_enabled=true is not supported

What it means

SelfAttention::new explicitly rejects configs with kv_cache_enabled set to true — the KV-cache attention path is not implemented in the candle metavoice port. This is a hard unsupported-feature guard, not a transient failure.

Source

Thrown at candle-transformers/src/models/metavoice.rs:449

    }

    // https://github.com/metavoiceio/metavoice-src/blob/11550bb4e8a1ad032cc1556cc924f7a4e767cbfa/fam/llm/layers/attn.py#L18
    struct SelfAttention {
        c_attn: Linear,
        c_proj: Linear,
        n_head: usize,
        span: tracing::Span,
    }

    impl SelfAttention {
        fn new(cfg: &Config, vb: VarBuilder) -> Result<Self> {
            // The different attention variants are likely to be identical but still we only accept
            // TorchAttn for now.
            if cfg.attn_kernel_type != AttnKernelType::TorchAttn {
                candle::bail!("only TorchAttn is supported")
            }
            if cfg.kv_cache_enabled {
                candle::bail!("kv_cache_enabled=true is not supported")
            }
            let c_attn = linear_b(cfg.n_embd, cfg.n_embd * 3, cfg.bias, vb.pp("c_attn"))?;
            let c_proj = linear_b(cfg.n_embd, cfg.n_embd, cfg.bias, vb.pp("c_proj"))?;
            Ok(Self {
                c_attn,
                c_proj,
                n_head: cfg.n_head,
                span: tracing::span!(tracing::Level::TRACE, "self-attn"),
            })
        }
    }

    impl Module for SelfAttention {
        fn forward(&self, xs: &Tensor) -> Result<Tensor> {
            let _enter = self.span.enter();
            let (b, t, c) = xs.dims3()?;
            let c_x = xs
                .apply(&self.c_attn)?

View on GitHub (pinned to d5fee525bf)

Solutions

Set kv_cache_enabled to false in the Config before constructing the model
Edit the model's config file to remove or disable the kv cache option and reload
Run generation without KV caching (accepting slower inference) since candle does not implement this path

Example fix

// before
let cfg = Config { kv_cache_enabled: true, .. };
// after
let cfg = Config { kv_cache_enabled: false, .. };

Defensive patterns

Strategy: validation

Validate before calling

if cfg.kv_cache_enabled {
    return Err("kv_cache_enabled must be false for metavoice in candle");
}

Try / catch

let model = Metavoice::new(&cfg, vb)
    .map_err(|e| format!("kv-cache unsupported: {e}"))?;

Prevention

Force kv_cache_enabled=false when deserializing Config
Do not enable KV cache options ported from non-candle configs
Review config.json for kv-cache flags before model construction

When it happens

Trigger: Building the MetaVoice model with a Config where kv_cache_enabled == true, typically deserialized from a checkpoint's config.json that enables KV caching.

Common situations: Loading a checkpoint whose upstream config enables KV cache for inference speed; enabling KV cache manually to try to speed up generation; reusing a config from a framework that supports KV cache.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/5410d019e75c02d7. Report an issue: GitHub.