{"record":{"id":"92503cf08e03e8a5","repo":"hiyouga/LlamaFactory","slug":"bf16-and-fp16-cannot-be-both-true","errorCode":null,"errorMessage":"bf16 and fp16 cannot be both True.","messagePattern":"bf16 and fp16 cannot be both True\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/megatron_merge.py","lineNumber":48,"sourceCode":"\ndef convert_mca_to_hf(\n    checkpoint_path: str,\n    output_path: str = \"./output\",\n    bf16: bool = False,\n    fp16: bool = False,\n    convert_model_max_length: int | None = None,\n):\n    \"\"\"Convert megatron checkpoint to HuggingFace format.\n\n    Args:\n        checkpoint_path: Path to the checkpoint to convert\n        output_path: Path to save the converted checkpoint\n        bf16: Use bfloat16 precision\n        fp16: Use float16 precision\n        convert_model_max_length: Change the model_max_length in hf config.json\n    \"\"\"\n    if bf16 and fp16:\n        raise ValueError(\"bf16 and fp16 cannot be both True.\")\n\n    torch_dtype = None\n    if bf16:\n        torch_dtype = torch.bfloat16\n    elif fp16:\n        torch_dtype = torch.float16\n\n    convert_checkpoint_to_hf(checkpoint_path, output_path, torch_dtype=torch_dtype)\n\n    if convert_model_max_length is not None:\n        config = AutoConfig.from_pretrained(output_path, trust_remote_code=True)\n        config.model_max_length = convert_model_max_length\n        config.save_pretrained(output_path)\n\n\ndef convert(\n    checkpoint_path: str,\n    output_path: str = \"./output\",","sourceCodeStart":30,"sourceCodeEnd":66,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/scripts/megatron_merge.py#L30-L66","documentation":"In the ref-model-free branch of preference loss computation (finetuning_args.use_ref_model == False), LlamaFactory only implements the 'orpo' (odds_ratio_loss) and 'simpo' (simpo_loss) losses (trainer.py:201). Any other pref_loss value (e.g. 'sigmoid' DPO, 'ipo', 'kto') needs reference log-probs and therefore raises NotImplementedError.","triggerScenarios":"DPO training YAML with pref_loss: sigmoid (default DPO loss) or ipo while use_ref_model / ref-related flags disable the reference model path; compute_preference_loss dispatches to the no-ref branch and hits the else clause.","commonSituations":"Switching a working DPO config to reference-free training and forgetting that only ORPO/SimPO are reference-free; copying a simpo example then changing pref_loss back to sigmoid without re-enabling the ref model.","solutions":["Set `pref_loss: orpo` or `pref_loss: simpo` if you want reference-free training","Or keep your loss (e.g. sigmoid DPO) and enable the reference model (provide ref model args / do not disable use_ref_model) so the dpo_loss branch is taken"],"exampleFix":"# before (YAML)\nstage: dpo\npref_loss: sigmoid\n# ...ref model disabled -> NotImplementedError\n\n# after (option 1)\npref_loss: simpo\n# after (option 2)\npref_loss: sigmoid  # and provide/reference the ref model as in examples/train_lora/dpo_llama3.yaml (lora_plus/ref model setup)","handlingStrategy":"validation","validationCode":"REF_FREE_LOSSES = {'orpo', 'simpo'}\nassert finetuning_args.use_ref_model or finetuning_args.pref_loss in REF_FREE_LOSSES, (\n    f'loss {finetuning_args.pref_loss!r} needs a reference model; use one of {REF_FREE_LOSSES} for ref-free training'\n)","typeGuard":"def is_ref_free_loss(loss_type: str) -> bool:\n    return loss_type in {\"orpo\", \"simpo\"}","tryCatchPattern":"try:\n    run_dpo(train_args)\nexcept NotImplementedError as e:\n    if 'Unknown loss type' in str(e):\n        raise SystemExit('Set pref_loss to orpo/simpo, or enable a reference model for this loss') from e\n    raise","preventionTips":["Encode the rule 'reference-free training => pref_loss in {orpo, simpo}' in config templates","Validate YAML against the stage's supported pref_loss values before submitting jobs"],"tags":["dpo","loss-function","reference-model","config"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}