unslothai/unsloth · error · ValueError

Model '{self.active_model_name}' has no chat_template set in

Error message

Model '{self.active_model_name}' has no chat_template set in its tokenizer_config.json. This is usually a problem with the model's HuggingFace repository — it is missing a 'chat_template' key. Please use a model that includes a chat template, or manually set one via tokenizer.chat_template before inference.

What it means

Before formatting the chat prompt, the engine verifies tokenizer.chat_template is present; if missing it raises ValueError explaining the model's HuggingFace repo lacks a 'chat_template' key in tokenizer_config.json, and instructs to use a model with a template or set tokenizer.chat_template manually. A preceding step may have tried get_chat_template and failed (logged as a warning), making this the final refusal.

Source

Thrown at studio/backend/core/inference/inference.py:1204

                    model_info["chat_turn_end_eos_ids"] = sorted(set(existing) | set(refreshed))
                except Exception as e:
                    logger.warning(f"Could not refresh chat turn-end eos after template: {e}")
            else:
                logger.info(
                    f"No registered Unsloth template for {self.active_model_name}, using tokenizer default"
                )
        except Exception as e:
            logger.warning(f"Could not apply get_chat_template: {e}")

        # Step 2: format with tokenizer.apply_chat_template().
        if system_prompt:
            template_messages = [{"role": "system", "content": system_prompt}] + messages
        else:
            template_messages = messages
        reasoning_channel_markers_resolved = False
        try:
            if not (hasattr(tokenizer, "chat_template") and tokenizer.chat_template):
                raise ValueError(
                    f"Model '{self.active_model_name}' has no chat_template set in its "
                    f"tokenizer_config.json. This is usually a problem with the model's "
                    f"HuggingFace repository — it is missing a 'chat_template' key. "
                    f"Please use a model that includes a chat template, or manually set "
                    f"one via tokenizer.chat_template before inference."
                )
            reasoning_channel_markers = None
            formatted_prompt = self._apply_chat_template_for_generation(
                tokenizer,
                template_messages,
                tools = tools,
                enable_thinking = enable_thinking,
                reasoning_effort = reasoning_effort,
                preserve_thinking = preserve_thinking,
                continue_final_message = continue_final_message,
            )

            # If tools were requested but the (possibly overridden) template ignored

View on GitHub (pinned to 203007d190)

Solutions

1. Switch to an instruct/chat-tuned variant of the model
2. Set a template manually: tokenizer.chat_template = "<known jinja template>" (or via the engine's custom-template hook if the earlier get_chat_template step provides one)
3. Upgrade transformers so chat_template.jinja sidecar files are read
4. Re-upload/fix the model repo's tokenizer_config.json to include the chat_template

Example fix

# before
load_model("meta-llama/Llama-3.1-8B")  # base, no template
# after
load_model("meta-llama/Llama-3.1-8B-Instruct")
# or
tokenizer.chat_template = CHATML_TEMPLATE

# After load, before first chat
tok = model_info["tokenizer"]
tok = getattr(tok, "tokenizer", tok)
if not (hasattr(tok, "chat_template") and tok.chat_template):
    tok.chat_template = FALLBACK_TEMPLATE  # or refuse early with a clear 409

def has_chat_template(tokenizer) -> bool:
    return bool(getattr(tokenizer, "chat_template", None))

try:
    formatted = tokenizer.apply_chat_template(msgs, ...)
except ValueError as e:
    if "no chat_template" in str(e):
        surface("use an instruct model or set tokenizer.chat_template", 409)
    raise

When it happens

Trigger: Loading a base model (often shipped without chat templates, e.g. base Llama/Qwen checkpoints) or a repo with a stripped tokenizer_config.json, then issuing a chat-format generation.

Common situations: User loads 'model-base' instead of 'model-instruct'; repo author removed the template; an older transformers version fails to read a template stored in a newer format (chat_template.jinja); custom fine-tunes uploaded without tokenizer config updates.

Related errors

• apply_chat_template_for_generation: no attempt produced a re
• no attempt rendered the continuation prefix
• the template produced an empty prompt
• apply_chat_template returned None — tokenizer may be incompa
• This execution artifact is outside the Recipe Studio dataset

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/5238878705a8feef. Report an issue: GitHub.