{"record":{"id":"9a6e2aebaee15d7e","repo":"hiyouga/LlamaFactory","slug":"training-render-expects-the-last-message-to-be-the","errorCode":null,"errorMessage":"training render expects the last message to be the supervised assistant turn; multi-turn conversations are split per turn in process_samples.","messagePattern":"training render expects the last message to be the supervised assistant turn; multi-turn conversations are split per turn in process_samples\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/core/rendering/rendering.py","lineNumber":145,"sourceCode":"                mm_type_ids = [0] * len(input_ids)\n            mm_type_ids = [marker if tid == token_id else t for t, tid in zip(mm_type_ids, input_ids)]\n\n        if mm_type_ids is not None:\n            result[\"mm_token_type_ids\"] = mm_type_ids\n\n    if is_generate:\n        # Generation prompt only -- nothing is supervised.\n        result = ModelInput(\n            input_ids=input_ids,\n            attention_mask=[1] * n,\n            labels=[IGNORE_INDEX] * n,\n            loss_weights=[0.0] * n,\n        )\n        _attach_multimodal(result)\n        return result\n\n    if not messages or messages[-1][\"role\"] != \"assistant\":\n        raise ValueError(\n            \"training render expects the last message to be the supervised assistant turn; \"\n            \"multi-turn conversations are split per turn in process_samples.\"\n        )\n\n    prompt_ids, _ = _encode(hf_messages[:-1], messages[:-1], add_generation_prompt=True)\n    if input_ids[: len(prompt_ids)] != prompt_ids:\n        # The prompt must be a token-prefix of the full sequence for the diff to be valid. If a\n        # template re-renders earlier turns when the final turn is appended, fail loud rather than\n        # mislabel.\n        raise ValueError(\n            \"prompt is not a token-prefix of the full sequence; the chat template is not \"\n            \"prefix-stable for this turn, so diff-based labeling is unsafe.\"\n        )\n\n    weight = messages[-1].get(\"loss_weight\", 1.0)\n    supervised = weight > 1e-6\n    labels = [IGNORE_INDEX] * len(prompt_ids)\n    loss_weights = [0.0] * len(prompt_ids)","sourceCodeStart":127,"sourceCodeEnd":163,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/core/rendering/rendering.py#L127-L163","documentation":"In training mode (is_generate=False) the renderer supervises exactly the final assistant turn, so the message list must end with role=='assistant'. Multi-turn supervision is expected to be pre-split per turn in process_samples; a list ending in a user/system/tool message has nothing to label and is rejected.","triggerScenarios":"Calling the training render path with messages whose last element is a 'user', 'system', or 'tool' turn, or an empty list; feeding an unsplit multi-turn conversation directly instead of the per-turn splits produced by process_samples.","commonSituations":"Custom datasets that end conversations with a tool response awaiting the model; bypassing the standard data pipeline and calling the renderer directly; off-by-one when slicing turns.","solutions":["Ensure each training sample's message list terminates with the assistant turn being supervised","Use the standard process_samples path, which splits multi-turn conversations per assistant turn","If the final turn is a tool response, append the assistant turn that should follow before rendering, or drop the trailing non-assistant turns"],"exampleFix":"# before\nmessages = [\n  {\"role\": \"user\", \"content\": [...]},\n  {\"role\": \"assistant\", \"content\": [...]},\n  {\"role\": \"tool\", \"content\": [...]},   # last turn is tool -> ValueError\n]\n\n# after\nmessages = [\n  {\"role\": \"user\", \"content\": [...]},\n  {\"role\": \"assistant\", \"content\": [...]},\n]\n# (supervise the earlier assistant turn; re-render the tool turn once an assistant reply exists)","handlingStrategy":"validation","validationCode":"def ends_with_assistant(messages: list[dict]) -> bool:\n    return bool(messages) and messages[-1].get(\"role\") == \"assistant\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Let process_samples split multi-turn data into per-assistant-turn samples","Assert the last role in a sample-level dataset validator"],"tags":["training-data","chat-template","supervision","rendering"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}