zylon-ai/private-gpt · error · ValueError

Tokenizer is required and must support apply_chat_template:

Error message

Tokenizer is required and must support apply_chat_template: {tokenizer}

What it means

ChatTemplatePromptStyle (the builtin 'chat' prompt style) builds prompts by calling tokenizer.apply_chat_template, so its constructor requires a tokenizer object that exposes that method. The constructor raises ValueError when tokenizer is None or when the supplied tokenizer lacks apply_chat_template (e.g. estimator or tiktoken tokenizers). This fails fast at wiring time instead of producing malformed prompts later.

Source

Thrown at private_gpt/components/llm/prompt_styles/chat_template_prompt_style.py:39

logger = logging.getLogger(__name__)

ChatTemplateContentFormat = Literal["string", "openai"]

_FALLBACK_USER_MESSAGE = {"role": "user", "content": ""}


class ChatTemplatePromptStyle(PromptStyleBase):
    def __init__(
        self,
        tokenizer: TokenizerBase | None = None,
        content_format: ChatTemplateContentFormat = "string",
        *args: Any,
        **kwargs: Any,
    ) -> None:
        super().__init__(*args, **kwargs)
        if tokenizer is None or not hasattr(tokenizer, "apply_chat_template"):
            raise ValueError(
                f"Tokenizer is required and must support apply_chat_template: {tokenizer}"
            )
        self._tokenizer = tokenizer
        self._content_format = content_format

    def _messages_to_prompt(
        self,
        messages: Sequence[ChatMessage],
        tools: Sequence[BaseTool] | None = None,
        reasoning_effort: ReasoningEffort | None = None,
        tokenize: bool = False,
        **kwargs: Any,
    ) -> PromptData:
        reasoning_effort = reasoning_effort or ReasoningEffort.NONE
        continue_final_message = (
            bool(messages) and messages[-1].role == MessageRole.ASSISTANT
        )
        conversation = self._to_hf_messages(messages)

View on GitHub (pinned to 4a030776a3)

Solutions

Pass a tokenizer that implements apply_chat_template (e.g. HuggingFaceTokenizer.from_pretrained(...)) into ChatTemplatePromptStyle / get_prompt_style('chat', tokenizer=...).
Align config: if tokenizer_mode is 'estimator' or 'tiktoken', either switch tokenizer_mode to 'huggingface'/'chat' or choose a different prompt_style.
If wiring manually, construct the tokenizer first and pass it: style = PromptStyleRegistry.get_prompt_style('chat', tokenizer=tok).

Example fix

// before
style = PromptStyleRegistry.get_prompt_style('chat', tokenizer=None)

// after
from private_gpt.components.llm.tokenizers.registry import TokenizerRegistry
tok = TokenizerRegistry.get_tokenizer('huggingface', model_id='mistralai/Mistral-7B-Instruct-v0.3')
style = PromptStyleRegistry.get_prompt_style('chat', tokenizer=tok)

Defensive patterns

Strategy: validation

Validate before calling

from private_gpt.components.llm.prompt_styles.chat_template_prompt_style import ChatTemplatePromptStyle

def can_build_chat_style(tokenizer) -> bool:
    return tokenizer is not None and hasattr(tokenizer, 'apply_chat_template')

if not can_build_chat_style(tok):
    raise ConfigurationError('prompt_style=chat requires an apply_chat_template-capable tokenizer')

Type guard

def is_chat_template_tokenizer(t: object) -> bool:
    return t is not None and callable(getattr(t, 'apply_chat_template', None))

Prevention

Keep prompt_style='chat' paired only with tokenizer modes that produce HF-style tokenizers ('huggingface'/'chat').
Validate tokenizer capability at startup wiring, not per request.
Write a smoke test that constructs the prompt style with your production tokenizer config.

When it happens

Trigger: Calling ChatTemplatePromptStyle(tokenizer=None) or PromptStyleRegistry.get_prompt_style('chat', tokenizer=<obj without apply_chat_template>). Typically happens when llm.tokenizer_mode is 'estimator'/'tiktoken' but prompt_style is 'chat', or when the DI wiring never passes the tokenizer into the prompt style.

Common situations: Config mismatch: prompt_style='chat' combined with a tokenizer mode that does not yield an HF-style tokenizer; forgetting to pass the tokenizer when constructing the style manually in tests or custom components; switching a deployment from a local model to an estimator-based setup without changing prompt_style.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/3c5ed3e077e43d54. Report an issue: GitHub.