What causes "Unsupported MM projector type: {}" in huggingface/candle?

Use a LLaVA checkpoint with a supported mm_projector_type (e.g. "mlp2x_gelu" or "identity")

Unsupported MM projector type: {}

Error message

Unsupported MM projector type: {}

What it means

Llava's multimodal projector builder supports only a fixed set of mm_projector_type values (e.g. mlp2x_gelu-like sequences and identity). Any other string in config.mm_projector_type falls through to this bail at load time.

Source

Thrown at candle-transformers/src/models/llava/mod.rs:119

                    config.hidden_size,
                    vb.pp("model.mm_projector.0"),
                )?);
                for i in 1..mlp_depth {
                    modules = modules.add(Activation::Gelu).add(linear(
                        config.hidden_size,
                        config.hidden_size,
                        vb.pp(format!("model.mm_projector.{}", i * 2)),
                    )?);
                }
                modules
            };
            Ok(Self { modules })
        } else if config.mm_projector_type == "identity" {
            Ok(Self {
                modules: seq().add(IdentityMap {}),
            })
        } else {
            bail!(
                "Unsupported MM projector type: {}",
                config.mm_projector_type
            )
        }
    }

    pub fn forward(&self, x: &Tensor) -> Result<Tensor> {
        self.modules.forward(x)
    }
}

pub struct ClipVisionTower {
    model: ClipVisionTransformer,
    select_layer: isize,
    select_feature_method: String,
    pub config: ClipVisionConfig,
}

View on GitHub (pinned to d5fee525bf)

Solutions

Use a LLaVA checkpoint with a supported mm_projector_type (e.g. "mlp2x_gelu" or "identity")
Check the match arms in llava/mod.rs load() for the exact supported strings and set config accordingly
Patch the projector builder to add the missing projector type

Example fix

// before (config.json)
{"mm_projector_type": "resampler", ...}
// after
{"mm_projector_type": "mlp2x_gelu", ...}

Defensive patterns

Strategy: validation

Validate before calling

let supported = ["mlp2x_gelu", "identity"]; // match load()'s accepted values
if !supported.contains(&config.mm_projector_type.as_str()) {
    return Err(format!("unsupported mm_projector_type: {}", config.mm_projector_type));
}

Type guard

fn has_supported_projector(c: &llava::Config) -> bool {
    matches!(c.mm_projector_type.as_str(), "mlp2x_gelu" | "identity")
}

Try / catch

match llava::MultimodalProjector::load(vb, config) {
    Err(e) if e.to_string().contains("Unsupported MM projector type") =>
        Err(anyhow!("use a LLaVA checkpoint with a supported projector type or patch load()")),
    r => r.map_err(Into::into),
}

Prevention

Check mm_projector_type in config.json before loading LLaVA checkpoints
Prefer LLaVA variants whose projector is mlp2x_gelu or identity with candle
Read the match arms in llava/mod.rs to know the current supported set

When it happens

Trigger: Loading a LLaVA checkpoint whose config.mm_projector_type is an unsupported string (e.g. "resampler", "lin", or a new transformers value) or the field missing/garbled so it matches neither supported branch.

Common situations: Using a LLaVA-NeXT/1.6 or variant with a projector type the candle port doesn't implement; typos in config; newer model releases ahead of the candle implementation.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/660037a42628b5a4. Report an issue: GitHub.