What causes "unsupported tokenizer model `{model_kind}`" in huggingface/candle?

Use a GGUF converted with a gpt2/BPE tokenizer, or convert the tokenizer to a standard tokenizer.json and load with tokenizers::Tokenizer::from_file instead

huggingface/candle · error

unsupported tokenizer model `{model_kind}`

Error message

unsupported tokenizer model `{model_kind}`

What it means

Tokenizer::from_gguf only supports GGUF tokenizers whose tokenizer.ggml.model is "gpt2" (the BPE-style layout candle knows how to rebuild). Any other tokenizer model kind stored in the GGUF bails with this message.

Source

Thrown at candle-core/src/quantized/tokenizer.rs:208

    let proc = tokenizers::processors::template::TemplateProcessing::builder()
        .try_single(single)
        .ok()?
        .try_pair(pair)
        .ok()?
        .special_tokens(specials)
        .build()
        .ok()?;

    Some(PostProcessorWrapper::Template(proc))
}

impl TokenizerFromGguf for Tokenizer {
    fn from_gguf(ct: &gguf_file::Content) -> Result<Self> {
        let model_kind = metadata_value(ct, "tokenizer.ggml.model")?
            .to_string()?
            .to_lowercase();
        if model_kind != "gpt2" {
            crate::bail!("unsupported tokenizer model `{model_kind}`");
        }

        let tokens = value_to_string_array(
            metadata_value(ct, "tokenizer.ggml.tokens")?,
            "tokenizer.ggml.tokens",
        )?;
        let vocab: Vocab = tokens
            .iter()
            .enumerate()
            .map(|(i, t)| (t.clone(), i as u32))
            .collect();
        let merges = merges_from_value(metadata_value(ct, "tokenizer.ggml.merges")?)?;

        let mut builder = BPE::builder().vocab_and_merges(vocab, merges);

        if let Ok(val) = metadata_value(ct, "tokenizer.ggml.unk_token_id") {
            let token_id = gguf_value_to_u32(val)?;
            if let Some(token) = tokens.get(token_id as usize) {

View on GitHub (pinned to d5fee525bf)

Solutions

Use a GGUF converted with a gpt2/BPE tokenizer, or convert the tokenizer to a standard tokenizer.json and load with tokenizers::Tokenizer::from_file instead
Re-export the model forcing tokenizer type gpt2 if the vocab is BPE-compatible
Extract tokens/merges from GGUF metadata yourself and construct the candle Tokenizer manually

Example fix

// before
let tok = Tokenizer::from_gguf(&ct)?;
// after
let tok = tokenizers::Tokenizer::from_file("tokenizer.json")?;

Defensive patterns

Strategy: validation

Validate before calling

let kind = ct.metadata.get("tokenizer.ggml.model")
    .map(|v| v.to_string())
    .transpose()?.unwrap_or_default().to_lowercase();
if kind != "gpt2" {
    // fall back to tokenizer.json instead of Tokenizer::from_gguf
}

Try / catch

match Tokenizer::from_gguf(&ct) {
    Ok(t) => t,
    Err(e) if e.to_string().contains("unsupported tokenizer model") => {
        tokenizers::Tokenizer::from_file("tokenizer.json")?.into()
    }
    Err(e) => return Err(e.into()),
}

Prevention

Check tokenizer.ggml.model in GGUF metadata before using from_gguf
Ship a separate tokenizer.json alongside GGUF models with non-BPE tokenizers
Prefer loading the tokenizer from tokenizer.json for LLaMA/SPM models

When it happens

Trigger: Calling Tokenizer::from_gguf (or QMatMul/model loaders that pull the tokenizer from GGUF content) on a file whose tokenizer.ggml.model metadata is not "gpt2", e.g. "llama" (sentencepiece/SPM) or "rwkv".

Common situations: Loading LLaMA-family GGUF models that use the SentencePiece tokenizer; older or alternative exporters writing non-gpt2 tokenizer models; expecting candle to auto-convert SPM tokenizers.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/22973cf9292b0ef9. Report an issue: GitHub.