{"record":{"id":"3f57439e4422b4c1","repo":"zylon-ai/private-gpt","slug":"cannot-set-add-generation-prompt-to-true-when-th","errorCode":null,"errorMessage":"Cannot set `add_generation_prompt` to True when the last message is from the assistant. Consider using `continue_final_message` instead.","messagePattern":"Cannot set `add_generation_prompt` to True when the last message is from the assistant\\. Consider using `continue_final_message` instead\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/llm/tokenizers/mistral.py","lineNumber":231,"sourceCode":"    Handles validation and formatting of messages and tools to ensure\n    compatibility with Mistral's requirements.\n    \"\"\"\n    tool_calls_module = _load_mistral_module(\n        \"mistral_common.protocol.instruct.tool_calls\"\n    )\n    Function = tool_calls_module.Function\n    Tool = tool_calls_module.Tool\n\n    if add_generation_prompt and continue_final_message:\n        raise ValueError(\n            \"Cannot set both `add_generation_prompt` and \"\n            \"`continue_final_message` to True.\"\n        )\n\n    last_message = messages[-1]\n\n    if add_generation_prompt and last_message[\"role\"] == \"assistant\":\n        raise ValueError(\n            \"Cannot set `add_generation_prompt` to True when \"\n            \"the last message is from the assistant. Consider \"\n            \"using `continue_final_message` instead.\"\n        )\n\n    if continue_final_message and last_message[\"role\"] != \"assistant\":\n        raise ValueError(\n            \"Cannot set `continue_final_message` to True when \"\n            \"the last message is not from the assistant.\"\n        )\n\n    # Mistral-common requires AssistantMessage content to be string\n    # https://github.com/mistralai/mistral-common/blob/f4a06998b75ed78bbf5aaf569590b772ea26c9f6/src/mistral_common/protocol/instruct/messages.py#L80\n    for message in messages:\n        # Remove reasoning as unsupported by Mistral\n        _ = message.pop(\"thinking\", None)\n\n        # Convert assistant message content to string if needed","sourceCodeStart":213,"sourceCodeEnd":249,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/llm/tokenizers/mistral.py#L213-L249","documentation":"Validation in _prepare_apply_chat_template_tools_and_messages: if add_generation_prompt=True, the last message must not already be an assistant message (otherwise the generation prompt would open a second assistant turn). The error text suggests continue_final_message for prefill-style usage.","triggerScenarios":"apply_chat_template(messages, add_generation_prompt=True) where messages[-1]['role'] == 'assistant' — i.e. assistant prefill / continuation inputs sent with the standard generation flag.","commonSituations":"Implementing assistant-prefill or few-shot examples that end with an assistant turn; replaying a full conversation (including prior assistant replies) and asking for the next answer without dropping the trailing assistant message.","solutions":["If you want to continue the assistant's text, use continue_final_message=True instead of add_generation_prompt.","If you want a fresh assistant answer, ensure the last message has role 'user' (or 'tool') before setting add_generation_prompt=True.","Drop or relabel trailing assistant messages when replaying history for generation."],"exampleFix":"# before: msgs ends with {'role':'assistant','content':'The answer is'}\nout = tok.apply_chat_template(msgs, add_generation_prompt=True)\n\n# after\nout = tok.apply_chat_template(msgs, continue_final_message=True)","handlingStrategy":"validation","validationCode":"def choose_flags(messages, want_prefill: bool) -> dict:\n    last_is_assistant = messages[-1]['role'] == 'assistant'\n    if want_prefill and not last_is_assistant:\n        raise ValueError('prefill requires trailing assistant message')\n    return {'add_generation_prompt': not want_prefill, 'continue_final_message': want_prefill}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Inspect messages[-1]['role'] before choosing the flag.","Use continue_final_message for assistant prefill; add_generation_prompt only after user/tool turns.","Trim trailing assistant messages when replaying history for fresh generation."],"tags":["mistral","chat-template","prefill","validation"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}