What causes "Unsupported projector activation: {}" in huggingface/candle?

Set projector_hidden_act to "gelu" or "relu" in the VoxtralConfig

Unsupported projector activation: {}

Error message

Unsupported projector activation: {}

What it means

The Voxtral multi-modal projector maps projector_hidden_act from the config to a candle activation, accepting only "gelu" and "relu"; anything else bails with 'Unsupported projector activation'. It mirrors the encoder activation whitelist but applies to the projector's hidden layer.

Source

Thrown at candle-transformers/src/models/voxtral/model.rs:706

impl VoxtralMultiModalProjector {
    pub fn new(cfg: &VoxtralConfig, vb: VarBuilder) -> Result<Self> {
        let linear_1 = linear_no_bias(
            cfg.audio_config.intermediate_size,
            cfg.text_config.hidden_size,
            vb.pp("linear_1"),
        )?;

        let linear_2 = linear_no_bias(
            cfg.text_config.hidden_size,
            cfg.text_config.hidden_size,
            vb.pp("linear_2"),
        )?;

        let activation = match cfg.projector_hidden_act.as_str() {
            "gelu" => candle_nn::Activation::Gelu,
            "relu" => candle_nn::Activation::Relu,
            _ => candle::bail!(
                "Unsupported projector activation: {}",
                cfg.projector_hidden_act
            ),
        };

        Ok(Self {
            linear_1,
            linear_2,
            activation,
        })
    }

    pub fn forward(&self, audio_features: &Tensor) -> Result<Tensor> {
        let x = self.linear_1.forward(audio_features)?;
        let x = x.apply(&self.activation)?;
        self.linear_2.forward(&x)
    }
}

View on GitHub (pinned to d5fee525bf)

Solutions

Set projector_hidden_act to "gelu" or "relu" in the VoxtralConfig
Patch the match in the projector constructor to support the activation your checkpoint uses (e.g. map "silu" to Activation::Silu)
Verify the expected activation from the model card/config.json and align the Rust config

Example fix

// before
let cfg = VoxtralConfig { projector_hidden_act: "silu".into(), .. };
// after
let cfg = VoxtralConfig { projector_hidden_act: "gelu".into(), .. };

Defensive patterns

Strategy: validation

Validate before calling

if !matches!(cfg.projector_hidden_act.as_str(), "gelu" | "relu") {
    return Err(anyhow::anyhow!("projector activation {:?} unsupported", cfg.projector_hidden_act));
}

Try / catch

match VoxtralModel::new(&cfg, vb) {
    Err(e) if e.to_string().contains("Unsupported projector activation") => {
        anyhow::bail!("set projector_hidden_act to gelu/relu or patch the projector")
    }
    r => r?,
}

Prevention

Validate projector_hidden_act alongside other config fields at load time
Alias HF activation names to candle Activation variants in one place
Confirm projector activation from the official Voxtral config before loading

When it happens

Trigger: Building the projector with a VoxtralConfig whose projector_hidden_act string is not "gelu" or "relu" (e.g. "silu", "gelu_new").

Common situations: Checkpoint config declaring silu for the projector; hand-ported Voxtral config; typo when constructing the config in Rust.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/499d091a8106b3a1. Report an issue: GitHub.