{"record":{"id":"0cfdfb50a3c354d0","repo":"huggingface/candle","slug":"mamba-layers-are-not-yet-supported-in-granitemoehy","errorCode":null,"errorMessage":"mamba layers are not yet supported in GraniteMoeHybrid inference","messagePattern":"mamba layers are not yet supported in GraniteMoeHybrid inference","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-transformers/src/models/granitemoehybrid.rs","lineNumber":554,"sourceCode":"        if cfg.layer_types.len() != cfg.num_hidden_layers {\n            candle::bail!(\n                \"layer_types length {} does not match num_hidden_layers {}\",\n                cfg.layer_types.len(),\n                cfg.num_hidden_layers\n            );\n        }\n        let blocks = cfg\n            .layer_types\n            .iter()\n            .enumerate()\n            .map(|(idx, layer_ty)| match layer_ty {\n                GraniteMoeHybridLayerType::Attention => {\n                    Block::load(vb.pp(format!(\"model.layers.{idx}\")), cfg)\n                }\n                GraniteMoeHybridLayerType::Mamba => {\n                    // TODO: Not supprting Mamba layers (blocks) for now,\n                    // so we only iterate over attention layers.\n                    candle::bail!(\n                        \"mamba layers are not yet supported in GraniteMoeHybrid inference\"\n                    )\n                }\n            })\n            .collect::<Result<Vec<_>>>()?;\n\n        Ok(Self {\n            word_token_embedding: wte,\n            blocks,\n            ln_f,\n            logits_scale: if cfg.logits_scaling == 0.0 {\n                1.0\n            } else {\n                1.0 / cfg.logits_scaling\n            },\n            embedding_scale: cfg.embedding_multiplier,\n        })\n    }","sourceCodeStart":536,"sourceCodeEnd":572,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-transformers/src/models/granitemoehybrid.rs#L536-L572","documentation":"GraniteMoeHybrid in candle only implements Attention blocks. When iterating layers, a layer typed Mamba hits a bail with a TODO — Mamba (SSM) blocks are not implemented, so a hybrid checkpoint containing them cannot run inference.","triggerScenarios":"Loading a GraniteMoeHybrid checkpoint whose layer_types includes GraniteMoeHybridLayerType::Mamba and calling load/forward.","commonSituations":"Using a newer/variant Granite hybrid model that mixes Mamba layers while the candle port only supports the attention-only variant; not filtering for attention-only checkpoints.","solutions":["Use a GraniteMoeHybrid checkpoint variant whose layer_types are all Attention","Wait for / upgrade to a candle version implementing Mamba blocks","Contribute or patch the Mamba block implementation in granitemoehybrid.rs"],"exampleFix":"// before (config.json)\n\"layer_types\": [\"attention\", \"mamba\", \"attention\", ...]\n// after (attention-only variant)\n\"layer_types\": [\"attention\", \"attention\", \"attention\", ...]","handlingStrategy":"validation","validationCode":"if cfg.layer_types.iter().any(|t| matches!(t, granitemoehybrid::GraniteMoeHybridLayerType::Mamba)) {\n    return Err(\"checkpoint contains Mamba layers, unsupported by candle GraniteMoeHybrid\");\n}","typeGuard":"fn is_attention_only(c: &granitemoehybrid::GraniteMoeHybridInternalConfig) -> bool {\n    c.layer_types.iter().all(|t| matches!(t, granitemoehybrid::GraniteMoeHybridLayerType::Attention))\n}","tryCatchPattern":"match granitemoehybrid::Model::new(&vb, &cfg) {\n    Err(e) if e.to_string().contains(\"mamba layers\") =>\n        Err(anyhow!(\"choose an attention-only GraniteMoeHybrid checkpoint or a different runtime\")),\n    r => r.map_err(Into::into),\n}","preventionTips":["Inspect layer_types in config.json before selecting this model implementation","Use attention-only Granite hybrid checkpoints with candle","Track candle releases for Mamba support or use a runtime that implements hybrid SSM layers"],"tags":["granitemoe","unsupported-feature","mamba"],"backgroundTag":"unsupported-architecture-config","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}