huggingface/candle · error

both hidden_act and hidden_activation are set

Error message

both hidden_act and hidden_activation are set

What it means

Gemma Config can specify the activation two ways: legacy hidden_act and newer hidden_activation. Config::hidden_act() rejects configs where both are set, because it cannot tell which one the checkpoint author intended. This is an eager config-validation error raised when building the model.

Source

Thrown at candle-transformers/src/models/gemma.rs:40

    pub hidden_activation: Option<Activation>,
    pub hidden_size: usize,
    pub intermediate_size: usize,
    pub num_attention_heads: usize,
    pub num_hidden_layers: usize,
    pub num_key_value_heads: usize,
    pub rms_norm_eps: f64,
    pub rope_theta: f64,
    pub vocab_size: usize,

    #[serde(default = "default_max_position_embeddings")]
    pub max_position_embeddings: usize,
}

impl Config {
    fn hidden_act(&self) -> Result<Activation> {
        match (self.hidden_act, self.hidden_activation) {
            (None, Some(act)) | (Some(act), None) => Ok(act),
            (Some(_), Some(_)) => candle::bail!("both hidden_act and hidden_activation are set"),
            (None, None) => candle::bail!("none of hidden_act and hidden_activation are set"),
        }
    }
}

#[derive(Debug, Clone)]
struct RmsNorm {
    weight: Tensor,
    eps: f64,
}

impl RmsNorm {
    fn new(dim: usize, eps: f64, vb: VarBuilder) -> Result<Self> {
        let weight = vb.get(dim, "weight")?;
        Ok(Self { weight, eps })
    }
}

View on GitHub (pinned to d5fee525bf)

Solutions

1. Remove one of the two keys from the config JSON (keep hidden_activation for newer checkpoints)
2. Ensure hidden_act is Option and defaults to None rather than being filled unconditionally
3. Pin to a candle version matching the checkpoint's config format

Example fix

// before (config.json)
{"hidden_act": "gelu", "hidden_activation": "gelu_pytorch_tanh", ...}
// after
{"hidden_activation": "gelu_pytorch_tanh", ...}

let both_set = cfg_json.get("hidden_act").is_some() && cfg_json.get("hidden_activation").is_some();
if both_set { return Err("config sets both hidden_act and hidden_activation"); }

fn activation_is_unambiguous(c: &gemma::Config) -> bool {
    c.hidden_act.is_some() ^ c.hidden_activation.is_some()
}

let model = gemma::Model::new(&vb, cfg).map_err(|e| {
    if e.to_string().contains("both hidden_act") {
        anyhow!("strip one activation key from config.json (prefer hidden_activation)")
    } else { e.into() }
});

When it happens

Trigger: Deserializing a Gemma config.json that contains both "hidden_act" and "hidden_activation" keys — typically a config from a newer transformers version fed to code that also fills the legacy field with a default.

Common situations: Mixing config versions (HuggingFace transformers renamed hidden_act to hidden_activation in newer Gemma revisions); hand-edited config.json; a serde default populating hidden_act when it should be None.

Related errors

• none of hidden_act and hidden_activation are set
• page_block_size {page_block_size_arg} does not match k shape
• GroupNorm: num_groups ({num_groups}) must divide num_channel
• unsupported 'value' data-type {} for {name}
• no graph defined in proto

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/aa752d030ccdb576. Report an issue: GitHub.