hiyouga/LlamaFactory · error · NotImplementedError

PPOTrainer does not support eval dataset yet.

Error message

PPOTrainer does not support eval dataset yet.

What it means

Error "PPOTrainer does not support eval dataset yet." thrown in hiyouga/LlamaFactory.

Source

Thrown at src/llamafactory/train/ppo/trainer.py:84

    def __init__(
        self,
        model_args: "ModelArguments",
        training_args: "Seq2SeqTrainingArguments",
        finetuning_args: "FinetuningArguments",
        generating_args: "GeneratingArguments",
        callbacks: Optional[list["TrainerCallback"]],
        model: "AutoModelForCausalLMWithValueHead",
        reward_model: Optional["AutoModelForCausalLMWithValueHead"],
        ref_model: Optional["AutoModelForCausalLMWithValueHead"],
        tokenizer: "PreTrainedTokenizer",
        processor: Optional["ProcessorMixin"],
        data_collator: "DataCollatorWithPadding",
        train_dataset: Optional["Dataset"] = None,
        eval_dataset: Optional["Dataset"] = None,
    ) -> None:
        if eval_dataset is not None:
            raise NotImplementedError("PPOTrainer does not support eval dataset yet.")

        # Check if TRL version is compatible (0.8.6 <= version <= 0.9.6)
        try:
            from transformers.utils.versions import require_version

            require_version(
                "trl>=0.8.6,<=0.9.6",
                "Incompatible TRL version detected. LLaMA-Factory ppo requires TRL version >=0.8.6,<=0.9.6. "
                f"Found version {trl_version}. Please install the correct version with: `pip install trl>=0.8.6,<=0.9.6`\n"
                "To fix: run `DISABLE_VERSION_CHECK=1 llamafactory-cli train example_ppo.yaml`\n",
            )
        except ImportError as e:
            raise e

        backward_batch_size = training_args.per_device_train_batch_size * training_args.gradient_accumulation_steps
        ppo_config = PPOConfig(
            model_name=model_args.model_name_or_path,
            learning_rate=training_args.learning_rate,

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove eval_dataset/val_size from the PPO config; evaluate the model separately using the SFT stage instead.

Example fix

# remove eval_dataset and val_size from the PPO YAML, then run a separate SFT eval job

When it happens

Trigger: Thrown at src/llamafactory/train/ppo/trainer.py:84 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/c762ea46c631d6ae. Report an issue: GitHub.