huggingface/candle · error

swiglu-multiple-of has to be set

Error message

swiglu-multiple-of has to be set

What it means

When the MLP non-linearity is Swiglu, the layer computes its hidden dimension by rounding down to a multiple of cfg.swiglu_multiple_of. If that config field is None, the computation is impossible, so new bails with this message. It is a missing-required-config-field error for Swiglu MLPs.

Source

Thrown at candle-transformers/src/models/metavoice.rs:515

    }

    impl MLP {
        fn new(cfg: &Config, vb: VarBuilder) -> Result<Self> {
            let hidden_dim = 4 * cfg.n_embd;
            let slf = match cfg.nonlinearity_type {
                NonLinearityType::Gelu => {
                    let c_fc = linear_b(cfg.n_embd, hidden_dim, cfg.bias, vb.pp("c_fc"))?;
                    let c_proj = linear_b(hidden_dim, cfg.n_embd, cfg.bias, vb.pp("c_proj"))?;
                    Self::Gelu {
                        c_fc,
                        c_proj,
                        span: tracing::span!(tracing::Level::TRACE, "mlp-gelu"),
                    }
                }
                NonLinearityType::Swiglu => {
                    let hidden_dim = (2 * hidden_dim) / 3;
                    let swiglu_multiple_of = match cfg.swiglu_multiple_of {
                        None => candle::bail!("swiglu-multiple-of has to be set"),
                        Some(smo) => smo,
                    };
                    let hidden_dim = swiglu_multiple_of * (hidden_dim + swiglu_multiple_of - 1)
                        / swiglu_multiple_of;
                    let w1 = linear_b(cfg.n_embd, hidden_dim, cfg.bias, vb.pp("w1"))?;
                    let w3 = linear_b(cfg.n_embd, hidden_dim, cfg.bias, vb.pp("w3"))?;
                    let c_proj = linear_b(hidden_dim, cfg.n_embd, cfg.bias, vb.pp("c_proj"))?;
                    Self::Swiglu {
                        w1,
                        w3,
                        c_proj,
                        span: tracing::span!(tracing::Level::TRACE, "mlp-swiglu"),
                    }
                }
            };
            Ok(slf)
        }
    }

View on GitHub (pinned to d5fee525bf)

Solutions

  1. Set cfg.swiglu_multiple_of to the value used by the checkpoint (commonly 128) in the Config or source config.json
  2. Switch non_linearity_type to a supported variant that does not require this field if that matches the checkpoint
  3. Compare your config.json against the official MetaVoice config and add all missing Swiglu-related fields

Example fix

// before
let cfg = Config { non_linearity_type: NonLinearityType::Swiglu, swiglu_multiple_of: None, .. };
// after
let cfg = Config { non_linearity_type: NonLinearityType::Swiglu, swiglu_multiple_of: Some(128), .. };
Defensive patterns

Strategy: validation

Validate before calling

if cfg.non_linearity_type == NonLinearityType::Swiglu && cfg.swiglu_multiple_of.is_none() {
    return Err("swiglu_multiple_of must be set when using Swiglu");
}

Try / catch

let model = Metavoice::new(&cfg, vb)
    .map_err(|e| format!("config incomplete: {e}"))?;

Prevention

When it happens

Trigger: Building the MetaVoice MLP with non_linearity_type = Swiglu while cfg.swiglu_multiple_of is None (field absent from the deserialized config.json or left unset in a hand-built Config).

Common situations: Checkpoint configs that omit swiglu_multiple_of because the original model used a different activation; manually constructed Config structs; partially converted configs between model versions.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/9a58f187c9b038e4. Report an issue: GitHub.