zylon-ai/private-gpt · error · ValueError

Cannot set `add_generation_prompt` to True when the last mes

Error message

Cannot set `add_generation_prompt` to True when the last message is from the assistant. Consider using `continue_final_message` instead.

What it means

Validation in _prepare_apply_chat_template_tools_and_messages: if add_generation_prompt=True, the last message must not already be an assistant message (otherwise the generation prompt would open a second assistant turn). The error text suggests continue_final_message for prefill-style usage.

Source

Thrown at private_gpt/components/llm/tokenizers/mistral.py:231

    Handles validation and formatting of messages and tools to ensure
    compatibility with Mistral's requirements.
    """
    tool_calls_module = _load_mistral_module(
        "mistral_common.protocol.instruct.tool_calls"
    )
    Function = tool_calls_module.Function
    Tool = tool_calls_module.Tool

    if add_generation_prompt and continue_final_message:
        raise ValueError(
            "Cannot set both `add_generation_prompt` and "
            "`continue_final_message` to True."
        )

    last_message = messages[-1]

    if add_generation_prompt and last_message["role"] == "assistant":
        raise ValueError(
            "Cannot set `add_generation_prompt` to True when "
            "the last message is from the assistant. Consider "
            "using `continue_final_message` instead."
        )

    if continue_final_message and last_message["role"] != "assistant":
        raise ValueError(
            "Cannot set `continue_final_message` to True when "
            "the last message is not from the assistant."
        )

    # Mistral-common requires AssistantMessage content to be string
    # https://github.com/mistralai/mistral-common/blob/f4a06998b75ed78bbf5aaf569590b772ea26c9f6/src/mistral_common/protocol/instruct/messages.py#L80
    for message in messages:
        # Remove reasoning as unsupported by Mistral
        _ = message.pop("thinking", None)

        # Convert assistant message content to string if needed

View on GitHub (pinned to 4a030776a3)

Solutions

1. If you want to continue the assistant's text, use continue_final_message=True instead of add_generation_prompt.
2. If you want a fresh assistant answer, ensure the last message has role 'user' (or 'tool') before setting add_generation_prompt=True.
3. Drop or relabel trailing assistant messages when replaying history for generation.

Example fix

# before: msgs ends with {'role':'assistant','content':'The answer is'}
out = tok.apply_chat_template(msgs, add_generation_prompt=True)

# after
out = tok.apply_chat_template(msgs, continue_final_message=True)

def choose_flags(messages, want_prefill: bool) -> dict:
    last_is_assistant = messages[-1]['role'] == 'assistant'
    if want_prefill and not last_is_assistant:
        raise ValueError('prefill requires trailing assistant message')
    return {'add_generation_prompt': not want_prefill, 'continue_final_message': want_prefill}

When it happens

Trigger: apply_chat_template(messages, add_generation_prompt=True) where messages[-1]['role'] == 'assistant' — i.e. assistant prefill / continuation inputs sent with the standard generation flag.

Common situations: Implementing assistant-prefill or few-shot examples that end with an assistant turn; replaying a full conversation (including prior assistant replies) and asking for the next answer without dropping the trailing assistant message.

Related errors

• Cannot set `continue_final_message` to True when the last me
• Cannot set both `add_generation_prompt` and `continue_final_
• mistral-common only supports function tools.
• Invalid system specification (dict): {system}
• Invalid system item in list (dict): {item}

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/3f57439e4422b4c1. Report an issue: GitHub.