{"record":{"id":"d1adaeb23cc1a0e2","repo":"hiyouga/LlamaFactory","slug":"dpo-training-requires-pair-format-samples-containi","errorCode":null,"errorMessage":"DPO training requires pair-format samples containing chosen/rejected responses. First sample from dataset '{dataset_name}' has keys: {sample_keys}. Please use pair data (e.g. a dataset with chosen_messages/rejected_messages).","messagePattern":"DPO training requires pair-format samples containing chosen/rejected responses\\. First sample from dataset '(.+?)' has keys: (.+?)\\. Please use pair data \\(e\\.g\\. a dataset with chosen_messages/rejected_messages\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/trainers/dpo_trainer.py","lineNumber":81,"sourceCode":"    rejected_logratios = policy_rejected_logps - ref_rejected_logps\n    logits = chosen_logratios - rejected_logratios\n    return -F.logsigmoid(beta * logits) * (1 - label_smoothing) - F.logsigmoid(-beta * logits) * label_smoothing\n\n\ndef _validate_dpo_dataset_format(train_dataset: DataEngine, dataset_path: str) -> None:\n    if train_dataset.streaming:\n        return\n\n    if len(train_dataset) == 0:\n        raise ValueError(f\"DPO training dataset is empty: {dataset_path}\")\n\n    sample = train_dataset[0]\n    if \"chosen_messages\" in sample and \"rejected_messages\" in sample:\n        return\n\n    dataset_name = sample.get(\"_dataset_name\", \"unknown\")\n    sample_keys = sorted(sample.keys())\n    raise ValueError(\n        \"DPO training requires pair-format samples containing chosen/rejected responses. \"\n        f\"First sample from dataset '{dataset_name}' has keys: {sample_keys}. \"\n        \"Please use pair data (e.g. a dataset with chosen_messages/rejected_messages).\"\n    )\n\n\nclass DPOTrainer(BaseTrainer):\n    def __init__(\n        self,\n        args: TrainingArguments,\n        model: HFModel,\n        renderer,\n        train_dataset,\n        callbacks=None,\n    ) -> None:\n        if args.cp_size > 1:\n            raise NotImplementedError(\"DPO trainer currently only supports cp_size == 1.\")\n","sourceCodeStart":63,"sourceCodeEnd":99,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/trainers/dpo_trainer.py#L63-L99","documentation":"ValueError from _validate_dpo_dataset_format (dpo_trainer.py:81) when the first sample lacks both 'chosen_messages' and 'rejected_messages' keys. DPO needs preference pairs; the message includes the offending dataset name and the sample's actual sorted keys so the mismatch is immediately visible. It is a format check on sample[0], so a single malformed leading row also triggers it.","triggerScenarios":"Feeding an SFT-style dataset (only 'messages' or instruction/output columns) to DPOTrainer; using a dataset whose pair columns are named differently and were not converted to chosen_messages/rejected_messages by the data converter.","commonSituations":"Copy-pasting an SFT yaml for DPO without changing the dataset; ranking data stored as separate chosen/rejected string columns without converter='pair'; a converter that silently skips conversion because column names don't match.","solutions":["Switch to a pair dataset that yields chosen_messages/rejected_messages (e.g. the ranking subset used in LlamaFactory examples).","If the raw data has chosen/rejected text columns, apply a pair converter so samples are emitted with chosen_messages/rejected_messages.","Print train_dataset[0] and compare its keys against the required pair keys to find naming mismatches.","Verify the dataset_info.json 'format'/'columns' mapping produces the *_messages fields."],"exampleFix":"# before: SFT dataset\n{\n  \"messages\": [{\"role\": \"user\", ...}, {\"role\": \"assistant\", ...}]\n}\n\n# after: DPO pair dataset\n{\n  \"chosen_messages\": [{\"role\": \"user\", ...}, {\"role\": \"assistant\", ...}],\n  \"rejected_messages\": [{\"role\": \"user\", ...}, {\"role\": \"assistant\", ...}]\n}","handlingStrategy":"validation","validationCode":"sample = train_dataset[0]\nassert \"chosen_messages\" in sample and \"rejected_messages\" in sample, f\"not pair data; keys={sorted(sample)}\"","typeGuard":"def is_pair_sample(sample: dict) -> bool:\n    return \"chosen_messages\" in sample and \"rejected_messages\" in sample","tryCatchPattern":null,"preventionTips":["Standardize preference datasets on chosen_messages/rejected_messages columns.","Add a unit test that asserts the first sample is pair-formatted."],"tags":["data","dataset","dpo","preference-data","validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}