What causes "new_decoder_architecture is not supported" in huggingface/candle?

Switch to a classic-architecture checkpoint such as tiiuae/falcon-7b or falcon-7b-instruct.

new_decoder_architecture is not supported

Error message

new_decoder_architecture is not supported

What it means

Falcon's Config::validate rejects new_decoder_architecture == true. The newer Falcon architecture (used by falcon-40b and later variants with parallel attention/MLP and multi-query layers) is not implemented in candle's Falcon model, so the config is rejected before construction.

Source

Thrown at candle-transformers/src/models/falcon.rs:79

            hidden_dropout: 0.0,
            attention_dropout: 0.0,
            n_head_kv: None,
            alibi: false,
            new_decoder_architecture: false,
            multi_query: true,
            parallel_attn: true,
            bias: false,
        }
    }
}

impl Config {
    pub fn validate(&self) -> Result<()> {
        if self.alibi {
            candle::bail!("alibi is not supported");
        }
        if self.new_decoder_architecture {
            candle::bail!("new_decoder_architecture is not supported");
        }
        if self.n_head_kv.is_some() {
            candle::bail!("n_head_kv is not supported");
        }
        Ok(())
    }

    // https://huggingface.co/tiiuae/falcon-7b/blob/main/config.json
    pub fn falcon7b() -> Self {
        // This is currently on par with the defaults, the defaults come from the Python default
        // arguments for the config initialization whereas the following come from the json config.
        Self {
            vocab_size: 65024,
            hidden_size: 4544,
            num_hidden_layers: 32,
            num_attention_heads: 71,
            layer_norm_epsilon: 1e-5,
            initializer_range: 0.02,

View on GitHub (pinned to d5fee525bf)

Solutions

Switch to a classic-architecture checkpoint such as tiiuae/falcon-7b or falcon-7b-instruct.
Check the model's config.json for new_decoder_architecture before downloading and pick a variant with it set to false.
If you need the new architecture, file/watch for upstream support in candle or implement the parallel-decoder path yourself.

Example fix

// before
let config = Config::from_json(&hf_hub_json("falcon-40b")?)?;
config.validate()?; // bails
// after
let config = Config::from_json(&hf_hub_json("falcon-7b")?)?;
config.validate()?; // ok

Defensive patterns

Strategy: validation

Validate before calling

if config.new_decoder_architecture {
    return Err(anyhow::anyhow!("new-architecture falcon (40b/180b) unsupported; use falcon-7b class models"));
}
config.validate()?;

Type guard

fn is_classic_falcon(c: &Config) -> bool { !c.new_decoder_architecture }

Try / catch

config.validate().map_err(|e| {
    if e.to_string().contains("new_decoder_architecture") {
        anyhow!("choose a classic-architecture falcon checkpoint (e.g. falcon-7b-instruct)")
    } else { e.into() }
})?;

Prevention

Verify new_decoder_architecture is false in config.json before using a checkpoint.
Prefer candle's built-in presets (Config::falcon7b()).
Track candle releases for new-architecture support before attempting larger falcon models.

When it happens

Trigger: Loading a Falcon config.json from a new-architecture model (e.g. falcon-40b, falcon-180b) and calling validate(), usually via main when running the falcon example or inference binary.

Common situations: Downloading the newest/largest Falcon checkpoint from HuggingFace and assuming candle supports it; only classic (falcon-7b style, new_decoder_architecture:false) checkpoints work.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/606a7aee21cacf4b. Report an issue: GitHub.