hiyouga/LlamaFactory · error · ValueError

training render expects the last message to be the supervise

Error message

training render expects the last message to be the supervised assistant turn; multi-turn conversations are split per turn in process_samples.

What it means

In training mode (is_generate=False) the renderer supervises exactly the final assistant turn, so the message list must end with role=='assistant'. Multi-turn supervision is expected to be pre-split per turn in process_samples; a list ending in a user/system/tool message has nothing to label and is rejected.

Source

Thrown at src/llamafactory/v1/core/rendering/rendering.py:145

                mm_type_ids = [0] * len(input_ids)
            mm_type_ids = [marker if tid == token_id else t for t, tid in zip(mm_type_ids, input_ids)]

        if mm_type_ids is not None:
            result["mm_token_type_ids"] = mm_type_ids

    if is_generate:
        # Generation prompt only -- nothing is supervised.
        result = ModelInput(
            input_ids=input_ids,
            attention_mask=[1] * n,
            labels=[IGNORE_INDEX] * n,
            loss_weights=[0.0] * n,
        )
        _attach_multimodal(result)
        return result

    if not messages or messages[-1]["role"] != "assistant":
        raise ValueError(
            "training render expects the last message to be the supervised assistant turn; "
            "multi-turn conversations are split per turn in process_samples."
        )

    prompt_ids, _ = _encode(hf_messages[:-1], messages[:-1], add_generation_prompt=True)
    if input_ids[: len(prompt_ids)] != prompt_ids:
        # The prompt must be a token-prefix of the full sequence for the diff to be valid. If a
        # template re-renders earlier turns when the final turn is appended, fail loud rather than
        # mislabel.
        raise ValueError(
            "prompt is not a token-prefix of the full sequence; the chat template is not "
            "prefix-stable for this turn, so diff-based labeling is unsafe."
        )

    weight = messages[-1].get("loss_weight", 1.0)
    supervised = weight > 1e-6
    labels = [IGNORE_INDEX] * len(prompt_ids)
    loss_weights = [0.0] * len(prompt_ids)

View on GitHub (pinned to f28afaf635)

Solutions

1. Ensure each training sample's message list terminates with the assistant turn being supervised
2. Use the standard process_samples path, which splits multi-turn conversations per assistant turn
3. If the final turn is a tool response, append the assistant turn that should follow before rendering, or drop the trailing non-assistant turns

Example fix

# before
messages = [
  {"role": "user", "content": [...]},
  {"role": "assistant", "content": [...]},
  {"role": "tool", "content": [...]},   # last turn is tool -> ValueError
]

# after
messages = [
  {"role": "user", "content": [...]},
  {"role": "assistant", "content": [...]},
]
# (supervise the earlier assistant turn; re-render the tool turn once an assistant reply exists)

def ends_with_assistant(messages: list[dict]) -> bool:
    return bool(messages) and messages[-1].get("role") == "assistant"

When it happens

Trigger: Calling the training render path with messages whose last element is a 'user', 'system', or 'tool' turn, or an empty list; feeding an unsplit multi-turn conversation directly instead of the per-turn splits produced by process_samples.

Common situations: Custom datasets that end conversations with a tool response awaiting the model; bypassing the standard data pipeline and calling the renderer directly; off-by-one when slicing turns.

Related errors

• prompt is not a token-prefix of the full sequence; the chat
• Not allowed
• special-token escape failed: the tokenizer normalized away t
• tool_call value is not valid JSON: {content['value']!r}
• tool_call must be a JSON object with 'name' and 'arguments'

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/9a6e2aebaee15d7e. Report an issue: GitHub.