{"record":{"id":"1b3697f9138da4ed","repo":"hiyouga/LlamaFactory","slug":"megatron-bridge-only-supports-the-pt-and-sft-s","errorCode":null,"errorMessage":"Megatron Bridge only supports the `pt` and `sft` stages.","messagePattern":"Megatron Bridge only supports the `pt` and `sft` stages\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":435,"sourceCode":"    # Check arguments\n    if finetuning_args.stage != \"sft\":\n        if training_args.predict_with_generate:\n            raise ValueError(\"`predict_with_generate` cannot be set as True except SFT.\")\n\n        if data_args.neat_packing:\n            raise ValueError(\"`neat_packing` cannot be set as True except SFT.\")\n\n        if data_args.train_on_prompt or data_args.mask_history:\n            raise ValueError(\"`train_on_prompt` or `mask_history` cannot be set as True except SFT.\")\n\n    if finetuning_args.stage == \"sft\" and training_args.do_predict and not training_args.predict_with_generate:\n        raise ValueError(\"Please enable `predict_with_generate` to save model predictions.\")\n\n    if finetuning_args.use_megatron_bridge:\n        if finetuning_args.use_mca or finetuning_args.use_hyper_parallel:\n            raise ValueError(\"Megatron Bridge cannot be used together with MCA or HyperParallel.\")\n        if finetuning_args.stage not in [\"pt\", \"sft\"]:\n            raise ValueError(\"Megatron Bridge only supports the `pt` and `sft` stages.\")\n        if finetuning_args.finetuning_type not in [\"full\", \"lora\"]:\n            raise ValueError(\"Megatron Bridge only supports `full` and `lora` finetuning.\")\n        if model_args.quantization_bit is not None:\n            raise ValueError(\"Quantized models are not supported with Megatron Bridge.\")\n        if training_args.deepspeed is not None:\n            raise ValueError(\"Megatron Bridge is incompatible with DeepSpeed.\")\n        if mb_args is None:\n            raise ValueError(\"Megatron Bridge arguments are missing. Please set USE_MEGATRON_BRIDGE=1.\")\n        _validate_megatron_bridge_parallel_args(mb_args, training_args.world_size)\n        finetuning_args.megatron_bridge_args = mb_args\n\n    if finetuning_args.stage in [\"rm\", \"ppo\"] and training_args.load_best_model_at_end:\n        raise ValueError(\"RM and PPO stages do not support `load_best_model_at_end`.\")\n\n    if finetuning_args.stage == \"ppo\":\n        if not training_args.do_train:\n            raise ValueError(\"PPO training does not support evaluation, use the SFT stage to evaluate models.\")\n","sourceCodeStart":417,"sourceCodeEnd":453,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L417-L453","documentation":"The Megatron Bridge backend only implements pretraining (pt) and supervised fine-tuning (sft). Preference tuning (dpo/kto), reward modeling (rm), and PPO have no Megatron training loop in this integration, so the parser rejects any other stage when use_megatron_bridge is true.","triggerScenarios":"USE_MEGATRON_BRIDGE=1 with stage: dpo (or rm, ppo, kto) in the train config; get_train_args() reaches the Megatron Bridge validation block and raises before parsing proceeds further.","commonSituations":"Enabling the bridge globally (exported env var) and then launching a DPO job from an unrelated config; assuming Megatron supports all LlamaFactory stages because the CLI accepts them.","solutions":["Change stage to sft or pt for Megatron Bridge runs.","Unset USE_MEGATRON_BRIDGE for DPO/RM/PPO/KTO jobs so they run on the standard HF trainer.","Keep per-stage launcher scripts/wrappers that set the right env var per task."],"exampleFix":"# before\nexport USE_MEGATRON_BRIDGE=1\n# cfg.yaml: stage: dpo\n\n# after (option 1: run DPO on standard backend)\nunset USE_MEGATRON_BRIDGE\n# cfg.yaml unchanged\n\n# after (option 2: stay on bridge)\n# cfg.yaml: stage: sft","handlingStrategy":"validation","validationCode":"import os\n\nif os.environ.get(\"USE_MEGATRON_BRIDGE\") == \"1\" and cfg.get(\"stage\", \"sft\") not in (\"pt\", \"sft\"):\n    raise SystemExit(\"Megatron Bridge supports only pt/sft stages; unset USE_MEGATRON_BRIDGE for this run\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Tag each config with the backend it targets and validate stage against backend in a preflight hook.","Keep separate launcher scripts per stage family (pretrain/fine-tune vs preference tuning)."],"tags":["megatron","config","stage","validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}