What causes "n_head_kv is not supported" in huggingface/candle?

Use a checkpoint without grouped-query attention (config.json has no n_head_kv or null), e.g. tiiuae/falcon-7b.

n_head_kv is not supported

Error message

n_head_kv is not supported

What it means

Falcon's Config::validate rejects configs where n_head_kv is Some. n_head_kv (grouped-query / multi-query attention) belongs to newer Falcon variants that candle's Falcon implementation does not support, so the config is refused up front.

Source

Thrown at candle-transformers/src/models/falcon.rs:82

            alibi: false,
            new_decoder_architecture: false,
            multi_query: true,
            parallel_attn: true,
            bias: false,
        }
    }
}

impl Config {
    pub fn validate(&self) -> Result<()> {
        if self.alibi {
            candle::bail!("alibi is not supported");
        }
        if self.new_decoder_architecture {
            candle::bail!("new_decoder_architecture is not supported");
        }
        if self.n_head_kv.is_some() {
            candle::bail!("n_head_kv is not supported");
        }
        Ok(())
    }

    // https://huggingface.co/tiiuae/falcon-7b/blob/main/config.json
    pub fn falcon7b() -> Self {
        // This is currently on par with the defaults, the defaults come from the Python default
        // arguments for the config initialization whereas the following come from the json config.
        Self {
            vocab_size: 65024,
            hidden_size: 4544,
            num_hidden_layers: 32,
            num_attention_heads: 71,
            layer_norm_epsilon: 1e-5,
            initializer_range: 0.02,
            use_cache: true,
            bos_token_id: 11,
            eos_token_id: 11,

View on GitHub (pinned to d5fee525bf)

Solutions

Use a checkpoint without grouped-query attention (config.json has no n_head_kv or null), e.g. tiiuae/falcon-7b.
Use Config::falcon7b() or another built-in preset that already passes validate().
If MQA is required, use a different candle model implementation that supports it or extend falcon.rs.

Example fix

// before: falcon-40b config.json with "n_head_kv": 8 -> validate() bails
// after: use a supported preset
let config = Config::falcon7b();
config.validate()?;

Defensive patterns

Strategy: validation

Validate before calling

if config.n_head_kv.is_some() {
    return Err(anyhow::anyhow!("checkpoint uses grouped-query attention (n_head_kv); not supported by candle falcon"));
}
config.validate()?;

Type guard

fn supports_no_mqa(c: &Config) -> bool { c.n_head_kv.is_none() }

Try / catch

if let Err(e) = config.validate() {
    if e.to_string().contains("n_head_kv") {
        anyhow::bail!("use a falcon checkpoint without grouped-query attention (e.g. falcon-7b)");
    }
    return Err(e.into());
}

Prevention

Check config.json for n_head_kv before downloading a falcon checkpoint.
Use falcon-7b-family checkpoints, which have no n_head_kv.
Route MQA/GQA requirements to a candle model that implements them.

When it happens

Trigger: Deserializing a Falcon config.json that contains a non-null "n_head_kv" (multi-query checkpoints like falcon-40b) and calling validate() before building the model.

Common situations: Selecting a Falcon checkpoint with grouped-query attention from HuggingFace; candle's supported falcon-7b config has n_head_kv: None.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/223b317e3865e6b2. Report an issue: GitHub.