{"record":{"id":"6f16e71373d80a65","repo":"hiyouga/LlamaFactory","slug":"megatron-bridge-cannot-be-used-together-with-mca-o","errorCode":null,"errorMessage":"Megatron Bridge cannot be used together with MCA or HyperParallel.","messagePattern":"Megatron Bridge cannot be used together with MCA or HyperParallel\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":433,"sourceCode":"        _set_transformers_logging()\n\n    # Check arguments\n    if finetuning_args.stage != \"sft\":\n        if training_args.predict_with_generate:\n            raise ValueError(\"`predict_with_generate` cannot be set as True except SFT.\")\n\n        if data_args.neat_packing:\n            raise ValueError(\"`neat_packing` cannot be set as True except SFT.\")\n\n        if data_args.train_on_prompt or data_args.mask_history:\n            raise ValueError(\"`train_on_prompt` or `mask_history` cannot be set as True except SFT.\")\n\n    if finetuning_args.stage == \"sft\" and training_args.do_predict and not training_args.predict_with_generate:\n        raise ValueError(\"Please enable `predict_with_generate` to save model predictions.\")\n\n    if finetuning_args.use_megatron_bridge:\n        if finetuning_args.use_mca or finetuning_args.use_hyper_parallel:\n            raise ValueError(\"Megatron Bridge cannot be used together with MCA or HyperParallel.\")\n        if finetuning_args.stage not in [\"pt\", \"sft\"]:\n            raise ValueError(\"Megatron Bridge only supports the `pt` and `sft` stages.\")\n        if finetuning_args.finetuning_type not in [\"full\", \"lora\"]:\n            raise ValueError(\"Megatron Bridge only supports `full` and `lora` finetuning.\")\n        if model_args.quantization_bit is not None:\n            raise ValueError(\"Quantized models are not supported with Megatron Bridge.\")\n        if training_args.deepspeed is not None:\n            raise ValueError(\"Megatron Bridge is incompatible with DeepSpeed.\")\n        if mb_args is None:\n            raise ValueError(\"Megatron Bridge arguments are missing. Please set USE_MEGATRON_BRIDGE=1.\")\n        _validate_megatron_bridge_parallel_args(mb_args, training_args.world_size)\n        finetuning_args.megatron_bridge_args = mb_args\n\n    if finetuning_args.stage in [\"rm\", \"ppo\"] and training_args.load_best_model_at_end:\n        raise ValueError(\"RM and PPO stages do not support `load_best_model_at_end`.\")\n\n    if finetuning_args.stage == \"ppo\":\n        if not training_args.do_train:","sourceCodeStart":415,"sourceCodeEnd":451,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L415-L451","documentation":"Megatron Bridge replaces the HuggingFace distributed stack with Megatron's, so it cannot coexist with MCA (Megatron-core via USE_MCA) or HyperParallel FSDP2, which are alternative distributed backends. The parser rejects the combination when finetuning_args.use_megatron_bridge is true and either use_mca or use_hyper_parallel is true.","triggerScenarios":"USE_MEGATRON_BRIDGE=1 in the environment while the config sets hyper_parallel/fspd2 options (use_hyper_parallel: true), or both USE_MCA and USE_MEGATRON_BRIDGE env vars set (USE_MCA is checked first, so this fires when mbridge won but the flag is still set), or the YAML itself sets use_megatron_bridge plus hyper-parallel flags.","commonSituations":"Shared cluster shells where USE_MCA or USE_MEGATRON_BRIDGE are exported globally in .bashrc; teams migrating between the three distributed backends leaving stale env vars; config templates that stack all performance flags.","solutions":["Unset conflicting backend env/flags: unset USE_MCA, and set hyper_parallel options off (use_hyper_parallel: false) in the config.","Pick exactly one backend per run: Megatron Bridge (USE_MEGATRON_BRIDGE=1), MCA (USE_MCA=1), or HF+HyperParallel (default path).","Audit exported variables before launching: env | grep -E 'USE_MCA|USE_MEGATRON_BRIDGE'."],"exampleFix":"# before\nexport USE_MCA=1\nexport USE_MEGATRON_BRIDGE=1\n# config: hyper_parallel: true\n\n# after (Megatron Bridge only)\nunset USE_MCA\nexport USE_MEGATRON_BRIDGE=1\n# config: hyper_parallel removed / false","handlingStrategy":"validation","validationCode":"import os\n\nbackends = sum([\n    os.environ.get(\"USE_MCA\") == \"1\",\n    os.environ.get(\"USE_MEGATRON_BRIDGE\") == \"1\",\n    bool(cfg.get(\"use_hyper_parallel\")),\n])\nif backends > 1:\n    raise SystemExit(\"Enable exactly one distributed backend: MCA, Megatron Bridge, or HyperParallel\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never export backend env vars globally; set them inline per launch command.","Add env | grep -E 'USE_MCA|USE_MEGATRON_BRIDGE' to launcher preflight."],"tags":["megatron","distributed","config","backend-conflict","validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}