unslothai/unsloth · error · ValueError

the template produced an empty prompt

Error message

the template produced an empty prompt

What it means

Part of the MLX backend's chat-template capability probe: it renders a minimal [{'role':'user','content':'hi'}] conversation through apply_chat_template_for_generation (against the processor for VLMs, the tokenizer otherwise) and requires a non-empty string. An empty/whitespace result means the model's chat template cannot render even a trivial text turn, so template-dependent features (tool calls, reasoning tags) would be garbage — hence ValueError.

Source

Thrown at studio/backend/core/inference/mlx_inference.py:1331

        Must use the same target the real request does. The recovery renderer
        returns None instead of raising for a model outside mlx-vlm's family
        list, so probing it would pass a template that cannot render at all.
        """
        from core.inference.chat_template_helpers import (
            apply_chat_template_for_generation,
            chat_render_target,
        )

        messages = [{"role": "user", "content": "hi"}]
        target = (
            chat_render_target(self._processor)
            if is_vision and self._processor is not None
            else self._tokenizer
        )
        rendered = apply_chat_template_for_generation(target, messages)
        if not rendered or not rendered.strip():
            raise ValueError("the template produced an empty prompt")
        return rendered

    def _populate_chat_template_info(
        self,
        model_name: str,
        native_template = _TEMPLATE_NOT_CAPTURED,
    ) -> None:
        """Mirror InferenceBackend._load_chat_template_info for MLX.

        Stores ``chat_template_info`` on ``self.models[model_name]``. The
        template recorded is the one the model shipped with, not an override:
        the capability classification and the editor's notion of "default"
        both read it, so an override installed on the tokenizer must not
        show up here."""
        entry = self.models.get(model_name)
        if not entry:
            return
        tok = entry.get("tokenizer")

View on GitHub (pinned to 203007d190)

Solutions

1. Use an instruct/chat-tuned variant of the model (repo names containing -Instruct / -it) which ships a valid chat_template.
2. Re-download the repo to rule out corrupt tokenizer files (clear the HF cache entry and reload).
3. Manually inspect tokenizer_config.json's chat_template field; if empty, supply a compatible template override or pick a different checkpoint.

Example fix

# before
backend.load('org/model-base')  # no chat template -> probe renders '' -> ValueError

# after
backend.load('org/model-base-instruct')  # ships valid chat_template

probe = apply_chat_template_for_generation(tokenizer, [{'role': 'user', 'content': 'hi'}])
if not probe or not probe.strip():
    mark_model_unsupported(model_name, reason='chat template renders empty')

try:
    template_probe = backend.probe_chat_template()
except ValueError as e:
    if 'empty prompt' in str(e):
        # load a fallback instruct checkpoint or reject the model in the UI
        raise ModelUnsupportedError(model_name) from e
    raise

When it happens

Trigger: Loading a model whose tokenizer/chat_template is missing, empty, raises internally and returns None, or renders to '' for plain text input; e.g. base models shipped without a chat_template, corrupted tokenizer files, or a template that only handles multimodal content while the probe targets the tokenizer.

Common situations: Raw pretrained checkpoints with no chat template; partially downloaded/corrupt tokenizer assets; exotic community repos with broken jinja templates.

Related errors

• apply_chat_template returned None — tokenizer may be incompa
• apply_chat_template_for_generation: no attempt produced a re
• no attempt rendered the continuation prefix
• Model '{self.active_model_name}' has no chat_template set in
• mlx-vlm's registered renderer returned an empty prompt.

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/483fa3c326523779. Report an issue: GitHub.