zylon-ai/private-gpt · error · ValueError

Cannot set both `add_generation_prompt` and `continue_final_

Error message

Cannot set both `add_generation_prompt` and `continue_final_message` to True.

What it means

Before formatting messages for mistral-common, _prepare_apply_chat_template_tools_and_messages validates mutually exclusive flags: add_generation_prompt (start a new assistant turn) and continue_final_message (extend the last assistant turn) cannot both be True, mirroring HF tokenizer semantics, and raises ValueError immediately.

Source

Thrown at private_gpt/components/llm/tokenizers/mistral.py:223

def _prepare_apply_chat_template_tools_and_messages(
    messages: list[dict[str, Any]],
    tools: list[dict[str, Any]] | None = None,
    continue_final_message: bool = False,
    add_generation_prompt: bool = False,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]] | None]:
    """Prepare messages and tools for Mistral's chat template format.

    Handles validation and formatting of messages and tools to ensure
    compatibility with Mistral's requirements.
    """
    tool_calls_module = _load_mistral_module(
        "mistral_common.protocol.instruct.tool_calls"
    )
    Function = tool_calls_module.Function
    Tool = tool_calls_module.Tool

    if add_generation_prompt and continue_final_message:
        raise ValueError(
            "Cannot set both `add_generation_prompt` and "
            "`continue_final_message` to True."
        )

    last_message = messages[-1]

    if add_generation_prompt and last_message["role"] == "assistant":
        raise ValueError(
            "Cannot set `add_generation_prompt` to True when "
            "the last message is from the assistant. Consider "
            "using `continue_final_message` instead."
        )

    if continue_final_message and last_message["role"] != "assistant":
        raise ValueError(
            "Cannot set `continue_final_message` to True when "
            "the last message is not from the assistant."
        )

View on GitHub (pinned to 4a030776a3)

Solutions

1. Set exactly one flag: add_generation_prompt=True for normal generation, or continue_final_message=True to extend the last assistant message.
2. Audit call sites that forward **kwargs into apply_chat_template and clamp/validate the pair there.
3. Add an assert/guard upstream so the invalid combination never reaches the tokenizer.

Example fix

# before
out = tok.apply_chat_template(msgs, add_generation_prompt=True, continue_final_message=True)

# after
out = tok.apply_chat_template(msgs, add_generation_prompt=True, continue_final_message=False)

def valid_template_flags(add_generation_prompt: bool, continue_final_message: bool) -> bool:
    return not (add_generation_prompt and continue_final_message)

assert valid_template_flags(agg, cont)

When it happens

Trigger: Calling the mistral tokenizer's apply_chat_template (messages, add_generation_prompt=True, continue_final_message=True) — usually a caller blindly forwarding both kwargs or copying defaults from another API.

Common situations: Adapters translating OpenAI-style 'continue' requests into chat-template calls; default-argument dicts shared across call sites; refactors that started passing continue_final_message without clearing add_generation_prompt.

Related errors

• Cannot set `add_generation_prompt` to True when the last mes
• Cannot set `continue_final_message` to True when the last me
• mistral-common only supports function tools.
• Mistral tokenizer must be in test mode. Set `mode=Validation
• Invalid system specification (dict): {system}

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/8136d2cddc7c69fd. Report an issue: GitHub.