huggingface/candle · error

layer_types length {} does not match num_hidden_layers {}

Error message

layer_types length {} does not match num_hidden_layers {}

What it means

GraniteMoeHybrid model load validates that cfg.layer_types (per-layer Attention/Mamba designation) has exactly num_hidden_layers entries before building the block list. A mismatch means the config is internally inconsistent and would index layer types incorrectly.

Source

Thrown at candle-transformers/src/models/granitemoehybrid.rs:537

        let x = x.i((.., seq_len - 1, ..))?.contiguous()?;
        // Project to vocabulary size
        let logits = x.matmul(&self.word_token_embedding.embeddings().t()?)?;
        let logits = logits.to_dtype(DType::F32)?;
        // Scale the logits if needed (that's also different from Granite 1)
        let scaled_logits = if (self.logits_scale - 1.0).abs() < f32::EPSILON {
            logits
        } else {
            logits.affine(self.logits_scale as f64, 0.)?
        };

        Ok(scaled_logits)
    }

    pub fn load(vb: VarBuilder, cfg: &GraniteMoeHybridInternalConfig) -> Result<Self> {
        let wte = embedding(cfg.vocab_size, cfg.hidden_size, vb.pp("model.embed_tokens"))?;
        let ln_f = RmsNorm::new(cfg.hidden_size, cfg.rms_norm_eps, vb.pp("model.norm"))?;
        if cfg.layer_types.len() != cfg.num_hidden_layers {
            candle::bail!(
                "layer_types length {} does not match num_hidden_layers {}",
                cfg.layer_types.len(),
                cfg.num_hidden_layers
            );
        }
        let blocks = cfg
            .layer_types
            .iter()
            .enumerate()
            .map(|(idx, layer_ty)| match layer_ty {
                GraniteMoeHybridLayerType::Attention => {
                    Block::load(vb.pp(format!("model.layers.{idx}")), cfg)
                }
                GraniteMoeHybridLayerType::Mamba => {
                    // TODO: Not supprting Mamba layers (blocks) for now,
                    // so we only iterate over attention layers.
                    candle::bail!(
                        "mamba layers are not yet supported in GraniteMoeHybrid inference"

View on GitHub (pinned to d5fee525bf)

Solutions

1. Make layer_types contain exactly num_hidden_layers entries
2. Derive layer_types programmatically: vec![layer_type; num_hidden_layers] or read all entries from config.json
3. Fix serde defaults so a missing layer_types doesn't silently produce an empty vec

Example fix

// before
let cfg = GraniteMoeHybridInternalConfig { num_hidden_layers: 40, layer_types: vec![], .. };
// after
let layer_types = vec![GraniteMoeHybridLayerType::Attention; cfg.num_hidden_layers];
let cfg = GraniteMoeHybridInternalConfig { num_hidden_layers: 40, layer_types, .. };

if cfg.layer_types.len() != cfg.num_hidden_layers {
    return Err(format!("layer_types len {} != num_hidden_layers {}", cfg.layer_types.len(), cfg.num_hidden_layers));
}

fn layer_types_consistent(c: &granitemoehybrid::GraniteMoeHybridInternalConfig) -> bool {
    c.layer_types.len() == c.num_hidden_layers
}

match granitemoehybrid::Model::new(&vb, &cfg) {
    Err(e) if e.to_string().contains("layer_types length") =>
        Err(anyhow!("fix layer_types in config to match num_hidden_layers")),
    r => r.map_err(Into::into),
}

When it happens

Trigger: Constructing GraniteMoeHybridInternalConfig with layer_types shorter/longer than num_hidden_layers — e.g. copying layer_types from a smaller checkpoint or forgetting to extend it when changing depth.

Common situations: Hand-building configs in code; deserializing a partial config.json where layer_types defaults to an empty vec; mixing configs across GraniteMoeHybrid model sizes.

Related errors

• GroupNorm: num_groups ({num_groups}) must divide num_channel
• page_block_size {page_block_size_arg} does not match k shape
• one_hot: index value {value} exceeds depth {depth}
• shape mismatch in layer-norm src: {:?} alpha: {:?} beta: {:?
• alibi is not supported

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/f1ab2f78fbb5469b. Report an issue: GitHub.