hiyouga/LlamaFactory · error · HTTPException

Not allowed

Error message

Not allowed

What it means

Megatron-Bridge datasets need assistant-only loss masks, expressed by wrapping assistant content in {% generation %}...{% endgeneration %}. _inject_generation_block (dataset_export.py:57) can only patch templates that branch with a literal `{% elif message['role'] == 'assistant' %}` block; if the chat template uses a different structure (loop over roles, set-based dispatch, filter blocks), injection is impossible and it raises ValueError.

Source

Thrown at src/llamafactory/api/app.py:104

    @app.get(
        "/v1/models",
        response_model=ModelList,
        status_code=status.HTTP_200_OK,
        dependencies=[Depends(verify_api_key)],
    )
    async def list_models():
        model_card = ModelCard(id=os.getenv("API_MODEL_NAME", "gpt-3.5-turbo"))
        return ModelList(data=[model_card])

    @app.post(
        "/v1/chat/completions",
        response_model=ChatCompletionResponse,
        status_code=status.HTTP_200_OK,
        dependencies=[Depends(verify_api_key)],
    )
    async def create_chat_completion(request: ChatCompletionRequest):
        if not chat_model.engine.can_generate:
            raise HTTPException(status_code=status.HTTP_405_METHOD_NOT_ALLOWED, detail="Not allowed")

        if request.stream:
            generate = create_stream_chat_completion_response(request, chat_model)
            return EventSourceResponse(generate, media_type="text/event-stream", sep="\n")
        else:
            return await create_chat_completion_response(request, chat_model)

    @app.post(
        "/v1/score/evaluation",
        response_model=ScoreEvaluationResponse,
        status_code=status.HTTP_200_OK,
        dependencies=[Depends(verify_api_key)],
    )
    async def create_score_evaluation(request: ScoreEvaluationRequest):
        if chat_model.engine.can_generate:
            raise HTTPException(status_code=status.HTTP_405_METHOD_NOT_ALLOWED, detail="Not allowed")

        return await create_score_evaluation_response(request, chat_model)

View on GitHub (pinned to f28afaf635)

Solutions

  1. Pass template: <name> in the YAML so a LlamaFactory registered template (which already formats correctly) is used instead of patching the tokenizer's native template
  2. Edit the tokenizer's chat_template.jinja to include an explicit `{% elif message['role'] == 'assistant' %}` branch, or pre-wrap assistant content with {% generation %} yourself
  3. Use a tokenizer whose template already contains {% generation %} blocks (mask-style templates)

Example fix

# before (chat_template.jinja)
{%- for message in messages %}{{ message.content }}{%- endfor %}
# no assistant elif branch -> ValueError

# after
{%- if message['role'] == 'user' %}{{ message.content }}
{%- elif message['role'] == 'assistant' %}{% generation %}{{ message.content }}{% endgeneration %}
{%- endif %}
Defensive patterns

Strategy: validation

Validate before calling

import re
ASSISTANT_ELIF = re.compile(r"{%\s*elif\s+message\['role'\]\s*==\s*'assistant'\s*%}")
GEN = re.compile(r"{%\s*generation\s*%}")
if not GEN.search(chat_template):
    assert ASSISTANT_ELIF.search(chat_template), (
        'Chat template lacks an assistant elif branch; provide `template:` or add {% generation %} manually'
    )

Type guard

def template_is_injectable(chat_template: str) -> bool:
    import re
    return bool(re.search(r"{%\s*generation\s*%}", chat_template)) or bool(
        re.search(r"{%\s*elif\s+message\['role'\]\s*==\s*'assistant'\s*%}", chat_template)
    )

Try / catch

try:
    build_chat_template_with_generation(tokenizer_path, template_name=template)
except ValueError as e:
    if 'Cannot inject' in str(e):
        raise SystemExit('Set `template:` in YAML or wrap assistant content with {% generation %} in the tokenizer template') from e
    raise

Prevention

When it happens

Trigger: Megatron Bridge PT/SFT run where the tokenizer's Jinja chat template has no `{% elif message['role'] == 'assistant' %}` branch and no existing {% generation %} block; the regex _ASSISTANT_ELIF_REGEX finds no match.

Common situations: Custom tokenizers with hand-written templates; vendor templates that iterate roles in a for-loop or use `{% if message.role == ... %}` (attribute access instead of ['role']) so the regex misses; templates already using a different masking convention.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/82eac34b05da105d. Report an issue: GitHub.