{"record":{"id":"e6341e47e680421f","repo":"hiyouga/LlamaFactory","slug":"ppo-training-does-not-support-evaluation-use-the","errorCode":null,"errorMessage":"PPO training does not support evaluation, use the SFT stage to evaluate models.","messagePattern":"PPO training does not support evaluation, use the SFT stage to evaluate models\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":452,"sourceCode":"        if finetuning_args.stage not in [\"pt\", \"sft\"]:\n            raise ValueError(\"Megatron Bridge only supports the `pt` and `sft` stages.\")\n        if finetuning_args.finetuning_type not in [\"full\", \"lora\"]:\n            raise ValueError(\"Megatron Bridge only supports `full` and `lora` finetuning.\")\n        if model_args.quantization_bit is not None:\n            raise ValueError(\"Quantized models are not supported with Megatron Bridge.\")\n        if training_args.deepspeed is not None:\n            raise ValueError(\"Megatron Bridge is incompatible with DeepSpeed.\")\n        if mb_args is None:\n            raise ValueError(\"Megatron Bridge arguments are missing. Please set USE_MEGATRON_BRIDGE=1.\")\n        _validate_megatron_bridge_parallel_args(mb_args, training_args.world_size)\n        finetuning_args.megatron_bridge_args = mb_args\n\n    if finetuning_args.stage in [\"rm\", \"ppo\"] and training_args.load_best_model_at_end:\n        raise ValueError(\"RM and PPO stages do not support `load_best_model_at_end`.\")\n\n    if finetuning_args.stage == \"ppo\":\n        if not training_args.do_train:\n            raise ValueError(\"PPO training does not support evaluation, use the SFT stage to evaluate models.\")\n\n        if model_args.shift_attn:\n            raise ValueError(\"PPO training is incompatible with S^2-Attn.\")\n\n        if finetuning_args.reward_model_type == \"lora\" and model_args.use_kt:\n            raise ValueError(\"KTransformers does not support lora reward model.\")\n\n        if finetuning_args.reward_model_type == \"lora\" and model_args.use_unsloth:\n            raise ValueError(\"Unsloth does not support lora reward model.\")\n\n        if training_args.report_to and any(\n            logger not in (\"wandb\", \"tensorboard\", \"trackio\", \"none\") for logger in training_args.report_to\n        ):\n            raise ValueError(\"PPO only accepts wandb, tensorboard, or trackio logger.\")\n\n    if not model_args.use_kt and training_args.parallel_mode == ParallelMode.NOT_DISTRIBUTED:\n        raise ValueError(\"Please launch distributed training with `llamafactory-cli` or `torchrun`.\")\n","sourceCodeStart":434,"sourceCodeEnd":470,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L434-L470","documentation":"PPO in LlamaFactory is implemented as an online training loop (policy + reward model + value model); there is no PPO 'evaluation' run type. If do_train is false (e.g. only do_eval/do_predict set), the parser raises this error and points you at the SFT stage for evaluating models.","triggerScenarios":"stage: ppo with do_train: false (or absent) and do_eval/do_predict: true in the training args — e.g. invoking a ppo config with --do_eval only.","commonSituations":"Trying to 'evaluate' a PPO-trained policy by running the ppo stage without training; CI pipelines that append --do_eval --eval_dataset ... to every config uniformly.","solutions":["Set do_train: true for PPO runs (that is the only supported mode).","To evaluate or chat with a PPO-trained checkpoint, switch stage: sft with the adapter/model path and use chat/api or do_eval there.","Remove --do_eval/--do_predict flags from ppo launches."],"exampleFix":"# before\nstage: ppo\ndo_train: false\ndo_eval: true\n\n# after\nstage: ppo\ndo_train: true\n# evaluation of the trained policy: use a separate sft/chat run with the saved adapter","handlingStrategy":"validation","validationCode":"if cfg.get(\"stage\") == \"ppo\" and not cfg.get(\"do_train\", False):\n    raise SystemExit(\"PPO requires do_train: true; evaluate trained policies via a separate sft/chat run\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat ppo configs as train-only; never append --do_eval/--do_predict to them.","Build a two-step eval workflow: train with ppo, then load the saved adapter in an sft/chat config for evaluation."],"tags":["config","ppo","validation","evaluation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}