{"record":{"id":"1aee8e47d2d65e1b","repo":"hiyouga/LlamaFactory","slug":"please-launch-distributed-training-with-llamafact","errorCode":null,"errorMessage":"Please launch distributed training with `llamafactory-cli` or `torchrun`.","messagePattern":"Please launch distributed training with `llamafactory-cli` or `torchrun`\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":469,"sourceCode":"        if not training_args.do_train:\n            raise ValueError(\"PPO training does not support evaluation, use the SFT stage to evaluate models.\")\n\n        if model_args.shift_attn:\n            raise ValueError(\"PPO training is incompatible with S^2-Attn.\")\n\n        if finetuning_args.reward_model_type == \"lora\" and model_args.use_kt:\n            raise ValueError(\"KTransformers does not support lora reward model.\")\n\n        if finetuning_args.reward_model_type == \"lora\" and model_args.use_unsloth:\n            raise ValueError(\"Unsloth does not support lora reward model.\")\n\n        if training_args.report_to and any(\n            logger not in (\"wandb\", \"tensorboard\", \"trackio\", \"none\") for logger in training_args.report_to\n        ):\n            raise ValueError(\"PPO only accepts wandb, tensorboard, or trackio logger.\")\n\n    if not model_args.use_kt and training_args.parallel_mode == ParallelMode.NOT_DISTRIBUTED:\n        raise ValueError(\"Please launch distributed training with `llamafactory-cli` or `torchrun`.\")\n\n    if training_args.deepspeed and training_args.parallel_mode != ParallelMode.DISTRIBUTED:\n        raise ValueError(\"Please use `FORCE_TORCHRUN=1` to launch DeepSpeed training.\")\n\n    if training_args.max_steps == -1 and data_args.streaming:\n        raise ValueError(\"Please specify `max_steps` in streaming mode.\")\n\n    if training_args.do_train and data_args.dataset is None:\n        raise ValueError(\"Please specify dataset for training.\")\n\n    if (training_args.do_eval or training_args.do_predict or training_args.predict_with_generate) and (\n        data_args.eval_dataset is None and data_args.val_size < 1e-6\n    ):\n        raise ValueError(\"Please make sure eval_dataset be provided or val_size >1e-6\")\n\n    if training_args.predict_with_generate:\n        if is_deepspeed_zero3_enabled():\n            raise ValueError(\"`predict_with_generate` is incompatible with DeepSpeed ZeRO-3.\")","sourceCodeStart":451,"sourceCodeEnd":487,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L451-L487","documentation":"Raised in parser.py:469 when training is launched in NOT_DISTRIBUTED mode while KTransformers (use_kt) is off. LlamaFactory requires its distributed launcher for training so that environment detection, device placement, and launcher patching work correctly; a bare `python` invocation bypasses that setup.","triggerScenarios":"Calling the training entry point directly, e.g. `python src/llamafactory/tuner.py ...` or a custom script that calls run_exp/run_sft without torchrun, so training_args.parallel_mode resolves to ParallelMode.NOT_DISTRIBUTED while model_args.use_kt is False.","commonSituations":"Porting LlamaFactory into a custom Python driver script; running the module via `python -m`; a wrapper (IDE run button, notebook) that spawns the process without the distributed environment variables.","solutions":["Launch with the CLI: `llamafactory-cli train config.yaml` (it auto-wraps with torchrun when needed)","Or launch explicitly with `FORCE_TORCHRUN=1 llamafactory-cli train config.yaml`","If using a custom script, import and call llamafactory.cli:main / launcher so parallel_mode is set up, instead of calling tuner functions raw"],"exampleFix":"# before\npython src/llamafactory/... my_train.py  # NOT_DISTRIBUTED -> raises\n\n# after\nllamafactory-cli train examples/train_lora/llama3_lora_sft.yaml","handlingStrategy":"validation","validationCode":"import torch.distributed as dist\nif not model_args.use_kt and not (dist.is_available() and dist.is_initialized()):\n    raise SystemExit(\"Launch via llamafactory-cli / torchrun, not bare python\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always launch training with llamafactory-cli or lmf, never a direct python entry","If embedding in a script, call the CLI via subprocess so launcher logic runs","Check RANK/WORLD_SIZE env vars exist before calling tuner APIs programmatically"],"tags":["launcher","distributed","torchrun","config-validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}