unslothai/unsloth · error · RuntimeError

apply_chat_template returned None — tokenizer may be incompa

Error message

apply_chat_template returned None — tokenizer may be incompatible

What it means

Raised when apply_chat_template_for_generation returns None for the MLX tokenizer path. Unlike an empty string (which template fallbacks may repair), None means the renderer outright failed — typically because the tokenizer has no usable chat template or is incompatible with the message/kwargs combination (tools, enable_thinking, reasoning_effort, continue_final_message). The backend treats None as fatal with this RuntimeError hinting at tokenizer incompatibility.

Source

Thrown at studio/backend/core/inference/mlx_inference.py:1541

        from mlx_lm.sample_utils import make_sampler, make_logits_processors

        from core.inference.chat_template_helpers import (
            apply_chat_template_for_generation,
            detect_think_prefill,
            render_with_native_template_fallback,
        )

        prompt = apply_chat_template_for_generation(
            self._tokenizer,
            messages,
            tools = tools,
            enable_thinking = enable_thinking,
            reasoning_effort = reasoning_effort,
            preserve_thinking = preserve_thinking,
            continue_final_message = continue_final_message,
        )
        if prompt is None:
            raise RuntimeError("apply_chat_template returned None — tokenizer may be incompatible")

        # Parity with the transformers backend: if the template dropped the
        # requested tools, fall back to the native template so MLX text models
        # keep advertising them. self._tokenizer is this entry's tokenizer, so
        # probe and native render share a renderer. (VLM renders via the
        # processor for image tokens and is not wired here.)
        model_info = self.models.get(self.active_model_name, {})
        render_result = render_with_native_template_fallback(
            formatted_prompt = prompt,
            tokenizer = self._tokenizer,
            model_info = model_info,
            active_model_name = self.active_model_name,
            messages = messages,
            tools = tools,
            enable_thinking = enable_thinking,
            reasoning_effort = reasoning_effort,
            preserve_thinking = preserve_thinking,
            continue_final_message = continue_final_message,

View on GitHub (pinned to 203007d190)

Solutions

1. Use a chat/instruct checkpoint with a complete chat_template that supports the features you pass (tools, thinking).
2. Drop the advanced kwargs (tools, enable_thinking, reasoning_effort, continue_final_message) and retry — if it renders, the template simply lacks those extensions.
3. Verify tokenizer/model provenance match; re-download the repo if tokenizer assets are suspect.

Example fix

# before
prompt = apply_chat_template_for_generation(tokenizer, messages,
    tools=tools, enable_thinking=True)  # returns None -> RuntimeError

# after
prompt = apply_chat_template_for_generation(tokenizer, messages)  # template lacks tool/thinking support

probe = apply_chat_template_for_generation(tokenizer, [{'role': 'user', 'content': 'hi'}])
if probe is None:
    raise ValueError('tokenizer cannot render chat; dropping advanced kwargs or using a chat model required')

try:
    prompt = apply_chat_template_for_generation(tokenizer, messages,
        tools=tools, enable_thinking=enable_thinking)
except RuntimeError as e:
    if 'returned None' in str(e) and tools is not None:
        prompt = apply_chat_template_for_generation(tokenizer, messages)  # retry without tools
    else:
        raise

When it happens

Trigger: Text-model generation where apply_chat_template returns None: missing chat_template in tokenizer_config.json, a template that raises on tool/thinking kwargs and the wrapper converts that to None, or a tokenizer from a different model family than the weights.

Common situations: Base models without chat templates; sending tools=... to a model whose template predates tool-calling syntax; mismatched tokenizer/model after manual conversions.

Related errors

• the template produced an empty prompt
• apply_chat_template_for_generation: no attempt produced a re
• no attempt rendered the continuation prefix
• Model '{self.active_model_name}' has no chat_template set in
• Unsloth MLX: use_adapter must be None, True, False, or a str

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/4e7815d15957eab5. Report an issue: GitHub.