hiyouga/LlamaFactory · error · ValueError

RM training dataset is empty: {dataset_path}

Error message

RM training dataset is empty: {dataset_path}

What it means

ValueError from _validate_rm_dataset_format (rm_trainer.py:35) when the reward-model train DataEngine has zero samples. Like the DPO counterpart, it is an early check at trainer construction that replaces a cryptic empty-dataloader failure. Unlike DPO, there is no streaming exemption in this function.

Source

Thrown at src/llamafactory/v1/trainers/rm_trainer.py:35

import torch.nn.functional as F

from ..accelerator.interface import Dim, DistributedInterface
from ..config import InputArgument, TrainingArguments, get_args
from ..config.arg_utils import ModelClass
from ..core.base_trainer import BaseTrainer
from ..core.data_engine import DataEngine
from ..core.model_engine import ModelEngine
from ..utils import logging
from ..utils.types import BatchInput, HFModel, Tensor


logger = logging.get_logger(__name__)


def _validate_rm_dataset_format(train_dataset: DataEngine, dataset_path: str) -> None:
    """Validate RM dataset format early for clearer error messages."""
    if len(train_dataset) == 0:
        raise ValueError(f"RM training dataset is empty: {dataset_path}")

    sample = train_dataset[0]
    if "chosen_messages" in sample and "rejected_messages" in sample:
        return

    dataset_name = sample.get("_dataset_name", "unknown")
    sample_keys = sorted(sample.keys())
    raise ValueError(
        "RM training requires pair-format samples containing chosen/rejected responses. "
        f"First sample from dataset '{dataset_name}' has keys: {sample_keys}. "
        "Please use pair data (e.g. a dataset with chosen_messages/rejected_messages, "
        "or set converter='pair' for raw chosen/rejected fields)."
    )


def _init_score_head(model: HFModel) -> None:
    """Initialize the score head for RM training with small Gaussian weights.

View on GitHub (pinned to f28afaf635)

Solutions

1. Load the dataset with the same configuration and assert len(train_dataset) > 0 before creating the trainer.
2. Check dataset_info.json and any filter/max_samples settings for over-aggressive filtering.
3. Confirm the data file path in dataset_info.json exists and contains rows.

assert len(train_dataset) > 0, f"RM dataset {dataset_path} is empty"

When it happens

Trigger: Constructing RMTrainer with an empty processed dataset: empty source file, all rows filtered out, or a dataset_info entry pointing at a nonexistent/empty data file.

Common situations: max_samples or filtering discards every row; wrong dataset name; converter column-name mismatch causing zero surviving samples during preprocessing.

Related errors

• RM training requires pair-format samples containing chosen/r
• DPO training dataset is empty: {dataset_path}
• DPO training requires pair-format samples containing chosen/
• Unknown mixing strategy: {data_args.mix_strategy}.
• Cannot specify `val_size` if `eval_dataset` is not None.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/b0d1d8ceace71748. Report an issue: GitHub.