huggingface/candle · error

none of hidden_act and hidden_activation are set

Error message

none of hidden_act and hidden_activation are set

What it means

Thrown when constructing Gemma activation if neither hidden_act nor hidden_activation is set in the config; at least one must be present to select the activation function.

Source

Thrown at candle-transformers/src/models/gemma.rs:41

    pub hidden_size: usize,
    pub intermediate_size: usize,
    pub num_attention_heads: usize,
    pub num_hidden_layers: usize,
    pub num_key_value_heads: usize,
    pub rms_norm_eps: f64,
    pub rope_theta: f64,
    pub vocab_size: usize,

    #[serde(default = "default_max_position_embeddings")]
    pub max_position_embeddings: usize,
}

impl Config {
    fn hidden_act(&self) -> Result<Activation> {
        match (self.hidden_act, self.hidden_activation) {
            (None, Some(act)) | (Some(act), None) => Ok(act),
            (Some(_), Some(_)) => candle::bail!("both hidden_act and hidden_activation are set"),
            (None, None) => candle::bail!("none of hidden_act and hidden_activation are set"),
        }
    }
}

#[derive(Debug, Clone)]
struct RmsNorm {
    weight: Tensor,
    eps: f64,
}

impl RmsNorm {
    fn new(dim: usize, eps: f64, vb: VarBuilder) -> Result<Self> {
        let weight = vb.get(dim, "weight")?;
        Ok(Self { weight, eps })
    }
}

impl Module for RmsNorm {

View on GitHub (pinned to d5fee525bf)

Solutions

1. Add "hidden_activation": "gelu_pytorch_tanh" (or the checkpoint's actual activation) to config.json
2. Set hidden_act in the Config struct when constructing it in code
3. Check the original model card / config.json from the HuggingFace repo for the correct value

Example fix

// before
let cfg = Config { hidden_act: None, hidden_activation: None, .. };
// after
let cfg = Config { hidden_act: None, hidden_activation: Some(Activation::GeluPytorchTanh), .. };

if cfg_json.get("hidden_act").is_none() && cfg_json.get("hidden_activation").is_none() {
    return Err("config must set hidden_act or hidden_activation");
}

fn has_activation(c: &gemma::Config) -> bool {
    c.hidden_act.is_some() || c.hidden_activation.is_some()
}

let model = gemma::Model::new(&vb, cfg).map_err(|e| {
    if e.to_string().contains("none of hidden_act") {
        anyhow!("add \"hidden_activation\" (e.g. gelu_pytorch_tanh) to config")
    } else { e.into() }
});

When it happens

Trigger: Loading a Gemma config.json missing both activation keys, or a struct constructed programmatically (e.g. Config { hidden_act: None, hidden_activation: None, .. }) without setting either.

Common situations: Hand-written or stripped config JSON; copy-pasted Config literal with activation fields left as None; a checkpoint variant whose config omits the key entirely.

Related errors

• both hidden_act and hidden_activation are set
• swiglu-multiple-of has to be set
• page_block_size {page_block_size_arg} does not match k shape
• GroupNorm: num_groups ({num_groups}) must divide num_channel
• no graph defined in proto

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/92ccefcd4a734459. Report an issue: GitHub.