BerriAI/litellm · error · Exception

No chat template found

Error message

No chat template found

What it means

Async HuggingFace template fetch (_afetch_and_extract_template): LiteLLM queried the model repo for a chat template and the result did not report success, so no template exists to render your messages. LiteLLM refuses to guess and raises this Exception. Without a chat template, a chat-formatted call to that model cannot be constructed.

Source

Thrown at litellm/litellm_core_utils/prompt_templates/factory.py:493

            bos_token = _extract_token_value(token_value=tokenizer_data.get("bos_token"))
            eos_token = _extract_token_value(token_value=tokenizer_data.get("eos_token"))
            chat_template = tokenizer_data["chat_template"]
        else:
            # Fallback: Try to fetch chat template from separate .jinja file
            template_result: Final = await get_template_fn(hf_model_name=model)
            if template_result.get("status") == "success":
                chat_template = template_result["chat_template"]
                # Still try to get tokens from tokenizer_config if available
                if (
                    tokenizer_config.get("status") == "success"
                    and "tokenizer" in tokenizer_config
                    and isinstance(tokenizer_config["tokenizer"], dict)
                ):
                    tokenizer_data: dict = tokenizer_config["tokenizer"]
                    bos_token = _extract_token_value(token_value=tokenizer_data.get("bos_token"))
                    eos_token = _extract_token_value(token_value=tokenizer_data.get("eos_token"))
            else:
                raise Exception("No chat template found")

    return chat_template, bos_token, eos_token


def _fetch_and_extract_template(
    model: str, chat_template: Any | None, get_config_fn, get_template_fn
) -> tuple[str, str, str]:
    """
    Sync version: Fetch template and tokens from HuggingFace.

    Returns: (chat_template, bos_token, eos_token)
    """
    from litellm.litellm_core_utils.prompt_templates.huggingface_template_handler import (
        _extract_token_value,
    )

    bos_token = ""
    eos_token = ""

View on GitHub (pinned to 6c2dcb801b)

Solutions

  1. Pass your own chat_template="<jinja2>" (or chat_template_file) so LiteLLM never needs to fetch one
  2. Switch to an instruct/chat-tuned model whose repo ships a chat template
  3. If the repo is gated/private, ensure HF credentials are configured so the fetch succeeds
  4. Verify the repo actually has a template: check tokenizer_config.json for 'chat_template' on huggingface.co

Example fix

# before
resp = await litellm.acompletion(
    model="huggingface/meta-llama/Llama-2-7b-hf",
    messages=[{"role": "user", "content": "Hello"}],
)

# after
resp = await litellm.acompletion(
    model="huggingface/meta-llama/Llama-2-7b-hf",
    messages=[{"role": "user", "content": "Hello"}],
    chat_template="{% for m in messages %}{{ bos_token }}[INST] {{ m['content'] }} [/INST]{% endfor %}",
)
Defensive patterns

Strategy: fallback

Validate before calling

import requests

def model_has_chat_template(repo_id: str) -> bool:
    cfg = requests.get(f"https://huggingface.co/{repo_id}/raw/main/tokenizer_config.json", timeout=10)
    if cfg.status_code != 200:
        return False
    return bool(cfg.json().get("chat_template"))

Try / catch

try:
    resp = await litellm.acompletion(model=hf_model, messages=messages)
except Exception as e:
    if "No chat template found" in str(e):
        resp = await litellm.acompletion(model=hf_model, messages=messages,
                                         chat_template=DEFAULT_CHAT_TEMPLATE)
    else:
        raise

Prevention

When it happens

Trigger: Calling acompletion() with a HuggingFace-hosted base model (no chat_template in tokenizer_config.json or chat_template.jinja), a typo'd/nonexistent repo name, or when HuggingFace returned an error/timeout payload treated as a failure.

Common situations: Using raw base models (e.g. Llama-2 style) that only have completion templates; private/gated repos where the anonymous fetch fails; network egress blocked so the HF fetch fails; repos that ship only a completion_format prompt.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/01071785822152cf. Report an issue: GitHub.