What causes "alibi is not supported" in huggingface/candle?

Use a checkpoint whose config.json has "alibi": false (e.g. tiiuae/falcon-7b or falcon-7b-instruct).

alibi is not supported

Error message

alibi is not supported

What it means

Falcon's Config::validate in candle-transformers rejects configurations with alibi set to true, because the candle Falcon implementation only supports rotary position embeddings (rotary) and does not implement ALiBi positional biases. bail! converts this into an immediate Err before any model weights are loaded.

Source

Thrown at candle-transformers/src/models/falcon.rs:76

            use_cache: true,
            bos_token_id: 11,
            eos_token_id: 11,
            hidden_dropout: 0.0,
            attention_dropout: 0.0,
            n_head_kv: None,
            alibi: false,
            new_decoder_architecture: false,
            multi_query: true,
            parallel_attn: true,
            bias: false,
        }
    }
}

impl Config {
    pub fn validate(&self) -> Result<()> {
        if self.alibi {
            candle::bail!("alibi is not supported");
        }
        if self.new_decoder_architecture {
            candle::bail!("new_decoder_architecture is not supported");
        }
        if self.n_head_kv.is_some() {
            candle::bail!("n_head_kv is not supported");
        }
        Ok(())
    }

    // https://huggingface.co/tiiuae/falcon-7b/blob/main/config.json
    pub fn falcon7b() -> Self {
        // This is currently on par with the defaults, the defaults come from the Python default
        // arguments for the config initialization whereas the following come from the json config.
        Self {
            vocab_size: 65024,
            hidden_size: 4544,
            num_hidden_layers: 32,

View on GitHub (pinned to d5fee525bf)

Solutions

Use a checkpoint whose config.json has "alibi": false (e.g. tiiuae/falcon-7b or falcon-7b-instruct).
If the config came from a local file, edit config.json to set alibi to false only if you have verified the weights are rotary-based; otherwise switch models.
Call config.validate() early in your own code (right after loading) so the mismatch fails fast with a clear message rather than mid-load.

Example fix

// before: loading a checkpoint with alibi: true
let config = Config::from_json(&json)?;
config.validate()?;
// after: pick a rotary-based Falcon variant
let config = Config::falcon7b(); // alibi: false, validated values
config.validate()?;

Defensive patterns

Strategy: validation

Validate before calling

let config = Config::from_json(&json)?;
if config.alibi {
    return Err(anyhow::anyhow!("checkpoint uses ALiBi; use a rotary Falcon checkpoint (alibi: false)"));
}
config.validate()?;

Type guard

fn supports_alibi_false(c: &Config) -> bool { !c.alibi }

Try / catch

match config.validate() {
    Ok(()) => load_model(config),
    Err(e) if e.to_string().contains("alibi") => eprintln!("switch to a rotary-based falcon checkpoint: {e}"),
    Err(e) => return Err(e.into()),
}

Prevention

Inspect the checkpoint's config.json (alibi field) before downloading.
Stick to known-supported checkpoints like tiiuae/falcon-7b(-instruct).
Call validate() immediately after config load, before allocating weights.

When it happens

Trigger: Deserializing a falcon Config (e.g. from a HuggingFace config.json) whose "alibi" field is true and then calling config.validate(), typically from main before building the model.

Common situations: Pointing candle at a Falcon variant that uses ALiBi (e.g. some falcon-40b style or fine-tuned checkpoints with alibi:true in config.json) instead of a rotary-based checkpoint like falcon-7b/instruct.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/592a137e3793ec7a. Report an issue: GitHub.