hiyouga/LlamaFactory · error · ValueError

PPO only accepts wandb, tensorboard, or trackio logger.

Error message

PPO only accepts wandb, tensorboard, or trackio logger.

What it means

This ValueError comes from the PPO-specific validation block in _check_args (src/llamafactory/hparams/parser.py:466). It rejects any `report_to` entry other than wandb, tensorboard, trackio, or the literal string "none" when the training stage is PPO. The restriction exists because TRL's PPO trainer only integrates with these logging backends.

Source

Thrown at src/llamafactory/hparams/parser.py:466

        raise ValueError("RM and PPO stages do not support `load_best_model_at_end`.")

    if finetuning_args.stage == "ppo":
        if not training_args.do_train:
            raise ValueError("PPO training does not support evaluation, use the SFT stage to evaluate models.")

        if model_args.shift_attn:
            raise ValueError("PPO training is incompatible with S^2-Attn.")

        if finetuning_args.reward_model_type == "lora" and model_args.use_kt:
            raise ValueError("KTransformers does not support lora reward model.")

        if finetuning_args.reward_model_type == "lora" and model_args.use_unsloth:
            raise ValueError("Unsloth does not support lora reward model.")

        if training_args.report_to and any(
            logger not in ("wandb", "tensorboard", "trackio", "none") for logger in training_args.report_to
        ):
            raise ValueError("PPO only accepts wandb, tensorboard, or trackio logger.")

    if not model_args.use_kt and training_args.parallel_mode == ParallelMode.NOT_DISTRIBUTED:
        raise ValueError("Please launch distributed training with `llamafactory-cli` or `torchrun`.")

    if training_args.deepspeed and training_args.parallel_mode != ParallelMode.DISTRIBUTED:
        raise ValueError("Please use `FORCE_TORCHRUN=1` to launch DeepSpeed training.")

    if training_args.max_steps == -1 and data_args.streaming:
        raise ValueError("Please specify `max_steps` in streaming mode.")

    if training_args.do_train and data_args.dataset is None:
        raise ValueError("Please specify dataset for training.")

    if (training_args.do_eval or training_args.do_predict or training_args.predict_with_generate) and (
        data_args.eval_dataset is None and data_args.val_size < 1e-6
    ):
        raise ValueError("Please make sure eval_dataset be provided or val_size >1e-6")

View on GitHub (pinned to f28afaf635)

Solutions

1. Set `report_to: wandb` (or `tensorboard` / `trackio`) in the PPO YAML config
2. To disable logging entirely, set `report_to: none` (or leave report_to empty/null)
3. Remove mlflow/clearml/aim or other unsupported entries from the config's report_to list

Example fix

# before (YAML)
stage: ppo
report_to:
  - mlflow

# after
stage: ppo
report_to: wandb

ALLOWED = {"wandb", "tensorboard", "trackio", "none"}
report_to = config.get("report_to") or []
if config.get("stage") == "ppo" and any(x not in ALLOWED for x in report_to):
    raise SystemExit(f"PPO only allows {sorted(ALLOWED)}; got {report_to}")

When it happens

Trigger: Running `llamafactory-cli train` with a config where finetuning_args.stage == "ppo" and training_args.report_to contains something else, e.g. report_to: ["mlflow"], ["clearml"], or ["codecarbon"]. The check `any(logger not in ("wandb","tensorboard","trackio","none") for logger in report_to)` fires.

Common situations: Copying an SFT config (which permits other loggers via the same field) and switching stage to ppo; enabling an experiment-tracking integration (mlflow/aim/clearml) globally in TrainingArguments defaults; HF `report_to="all"` resolving to multiple backends.

Related errors

• Invalid role
• `reward_model` is necessary for PPO training.
• Unknown logging level: {env_level_str}.
• `reward_model_type` cannot be lora for Freeze/Full PPO train
• `reward_model_type` cannot be oft for Freeze/Full PPO traini

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/7c940c4eac76f795. Report an issue: GitHub.