hiyouga/LlamaFactory · error · ValueError

Failed to inject {% generation %} into chat template.

Error message

Failed to inject {% generation %} into chat template.

What it means

Error "Failed to inject {% generation %} into chat template." thrown in hiyouga/LlamaFactory.

Source

Thrown at src/llamafactory/train/megatron_bridge/dataset_export.py:76

            "Cannot inject {% generation %} into chat template: "
            "no `{% elif message['role'] == 'assistant' %}` block found."
        )

    body = match.group(2)
    if _END_GENERATION_REGEX.search(body):
        return chat_template

    patched = (
        chat_template[: match.start()]
        + match.group(1)
        + "{% generation %}"
        + body
        + "{% endgeneration %}"
        + match.group(3)
        + chat_template[match.end() :]
    )
    if not _GENERATION_REGEX.search(patched):
        raise ValueError("Failed to inject {% generation %} into chat template.")
    return patched


def build_chat_template_with_generation(
    tokenizer_path: str | None,
    trust_remote_code: bool = False,
    template_name: str | None = None,
) -> str | None:
    r"""Build a chat template that supports assistant-only loss masks.

    Prefers the LLaMA-Factory registered template (same formatting as HF Trainer),
    then falls back to patching the tokenizer's native chat template.
    """
    if not tokenizer_path:
        return None

    try:
        from transformers import AutoTokenizer

View on GitHub (pinned to f28afaf635)

Solutions

1. Verify the chat template after injection contains matching {% generation %} and {% endgeneration %} tags; fix any custom template syntax so the injected block is valid Jinja.

Example fix

# wrap assistant content explicitly
{% generation %}...{% endgeneration %}

When it happens

Trigger: Thrown at src/llamafactory/train/megatron_bridge/dataset_export.py:76 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/8a78141760b835ce. Report an issue: GitHub.