zylon-ai/private-gpt · error · ValueError

Cannot set `continue_final_message` to True when the last me

Error message

Cannot set `continue_final_message` to True when the last message is not from the assistant.

What it means

The inverse rule of the prefill validation: continue_final_message=True requires the last message to be an assistant message, because it extends that message in place. A trailing user/tool message with this flag is contradictory and raises ValueError in _prepare_apply_chat_template_tools_and_messages.

Source

Thrown at private_gpt/components/llm/tokenizers/mistral.py:238

    Tool = tool_calls_module.Tool

    if add_generation_prompt and continue_final_message:
        raise ValueError(
            "Cannot set both `add_generation_prompt` and "
            "`continue_final_message` to True."
        )

    last_message = messages[-1]

    if add_generation_prompt and last_message["role"] == "assistant":
        raise ValueError(
            "Cannot set `add_generation_prompt` to True when "
            "the last message is from the assistant. Consider "
            "using `continue_final_message` instead."
        )

    if continue_final_message and last_message["role"] != "assistant":
        raise ValueError(
            "Cannot set `continue_final_message` to True when "
            "the last message is not from the assistant."
        )

    # Mistral-common requires AssistantMessage content to be string
    # https://github.com/mistralai/mistral-common/blob/f4a06998b75ed78bbf5aaf569590b772ea26c9f6/src/mistral_common/protocol/instruct/messages.py#L80
    for message in messages:
        # Remove reasoning as unsupported by Mistral
        _ = message.pop("thinking", None)

        # Convert assistant message content to string if needed
        if message.get("role") == "assistant":
            content = message.get("content")
            if isinstance(content, list):
                content = "\n".join(chunk.get("text") or "" for chunk in content)
                message["content"] = content

    # Mistral requires "parameters" and "description" to be present even if empty

View on GitHub (pinned to 4a030776a3)

Solutions

1. Only set continue_final_message=True when the conversation actually ends with an assistant message you want extended.
2. For normal generation over a user turn, use add_generation_prompt=True.
3. Derive the flag from the last message role at the call site instead of hard-coding it.

Example fix

# before
out = tok.apply_chat_template(msgs, continue_final_message=True)  # msgs[-1] is user

# after
flag = msgs[-1]['role'] == 'assistant'
out = tok.apply_chat_template(msgs, add_generation_prompt=not flag, continue_final_message=flag)

last = messages[-1]['role']
assert not (continue_final_message and last != 'assistant'), 'continue_final_message needs trailing assistant message'

When it happens

Trigger: apply_chat_template(messages, continue_final_message=True) where messages[-1]['role'] != 'assistant' (e.g. a normal user-turn conversation).

Common situations: Toggling continue_final_message based on a config flag while conversations still end with user messages; copy-pasting prefill code into a normal chat-completion path; client code defaulting the flag to True.

Related errors

• Cannot set `add_generation_prompt` to True when the last mes
• Cannot set both `add_generation_prompt` and `continue_final_
• mistral-common only supports function tools.
• Invalid system specification (dict): {system}
• Invalid system item in list (dict): {item}

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/07206f673d635b11. Report an issue: GitHub.