hiyouga/LlamaFactory · error · HTTPException

Invalid role

Error message

Invalid role

What it means

LlamaFactory's PPO trainer subclasses TRL 0.9.x's PPOTrainer, whose training loop never supported evaluation datasets; run_ppo therefore rejects eval_dataset up front (trainer.py:84) with NotImplementedError instead of silently ignoring or crashing mid-eval. This is a design limitation of the pinned TRL version, not a transient bug.

Source

Thrown at src/llamafactory/api/chat.py:105

        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid length")

    if request.messages[0].role == Role.SYSTEM:
        content = request.messages.pop(0).content
        if isinstance(content, list):
            system = content[0].text if content else ""
        else:
            system = content
    else:
        system = None

    if len(request.messages) % 2 == 0:
        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Only supports u/a/u/a/u...")

    input_messages = []
    images, videos, audios = [], [], []
    for i, message in enumerate(request.messages):
        if i % 2 == 0 and message.role not in [Role.USER, Role.TOOL]:
            raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid role")
        elif i % 2 == 1 and message.role not in [Role.ASSISTANT, Role.FUNCTION]:
            raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid role")

        if message.role == Role.ASSISTANT and isinstance(message.tool_calls, list) and len(message.tool_calls):
            tool_calls = [
                {"name": tool_call.function.name, "arguments": tool_call.function.arguments}
                for tool_call in message.tool_calls
            ]
            content = json.dumps(tool_calls, ensure_ascii=False)
            input_messages.append({"role": ROLE_MAPPING[Role.FUNCTION], "content": content})
        elif isinstance(message.content, list):
            text_content = ""
            for input_item in message.content:
                if input_item.type == "text":
                    text_content += input_item.text
                elif input_item.type == "image_url":
                    text_content += IMAGE_PLACEHOLDER
                    image_url = input_item.image_url.url

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove eval settings from the PPO config: set `val_size: 0` and drop eval_dataset entries
2. If periodic quality checks are needed, run separate inference/eval jobs (llamafactory-cli chat / eval) on checkpoints instead of in-training eval

Example fix

# before (YAML)
stage: ppo
val_size: 0.1  # creates eval_dataset -> NotImplementedError

# after
stage: ppo
val_size: 0

assert data_args.val_size == 0 and not data_args.eval_dataset, (
    'PPOTrainer does not support eval datasets; remove val_size/eval_dataset from PPO configs'
)

def ppo_config_is_eval_free(yaml_cfg: dict) -> bool:
    return float(yaml_cfg.get('val_size', 0) or 0) == 0 and not yaml_cfg.get('eval_dataset')

try:
    from llamafactory.train.ppo.workflow import run_ppo
    run_ppo(train_args)
except NotImplementedError as e:
    if 'eval dataset' in str(e):
        raise SystemExit('Remove val_size/eval_dataset from the PPO config') from e
    raise

When it happens

Trigger: A PPO YAML that sets `val_size: >0` / eval_dataset (e.g. copied from an SFT config with eval split), causing run_ppo to construct PPOTrainer with eval_dataset not None.

Common situations: Copying an SFT/DPO config with val_size for PPO; assuming all stages share the eval interface.

Related errors

• `reward_model` is necessary for PPO training.
• `reward_model_type` cannot be lora for Freeze/Full PPO train
• `reward_model_type` cannot be oft for Freeze/Full PPO traini
• Cannot use PiSSA for current training stage.
• RM and PPO stages do not support `load_best_model_at_end`.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/014d775d8a6ab50c. Report an issue: GitHub.