{"record":{"id":"a10e036324cfcf14","repo":"hiyouga/LlamaFactory","slug":"vllm-sglang-backend-is-only-available-for-api-cli","errorCode":null,"errorMessage":"vLLM/SGLang backend is only available for API, CLI and Web.","messagePattern":"vLLM/SGLang backend is only available for API, CLI and Web\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/parser.py","lineNumber":530,"sourceCode":"        if finetuning_args.use_badam:\n            if finetuning_args.badam_mode == \"ratio\":\n                raise ValueError(\"Radio-based BAdam does not yet support distributed training, use layer-wise BAdam.\")\n            elif not is_deepspeed_zero3_enabled():\n                raise ValueError(\"Layer-wise BAdam only supports DeepSpeed ZeRO-3 training.\")\n\n    if training_args.deepspeed is not None and (finetuning_args.use_galore or finetuning_args.use_apollo):\n        raise ValueError(\"GaLore and APOLLO are incompatible with DeepSpeed yet.\")\n\n    if (\n        not finetuning_args.use_mca\n        and not finetuning_args.use_megatron_bridge\n        and training_args.fp8\n        and model_args.quantization_bit is not None\n    ):\n        raise ValueError(\"FP8 training is not compatible with quantization. Please disable one of them.\")\n\n    if model_args.infer_backend != EngineName.HF:\n        raise ValueError(\"vLLM/SGLang backend is only available for API, CLI and Web.\")\n\n    if model_args.use_unsloth and is_deepspeed_zero3_enabled():\n        raise ValueError(\"Unsloth is incompatible with DeepSpeed ZeRO-3.\")\n\n    if model_args.use_kt and is_deepspeed_zero3_enabled():\n        raise ValueError(\"KTransformers is incompatible with DeepSpeed ZeRO-3.\")\n\n    _set_env_vars()\n    _verify_model_args(model_args, data_args, finetuning_args)\n    _check_extra_dependencies(model_args, finetuning_args, training_args)\n    _verify_trackio_args(training_args)\n\n    if (\n        not finetuning_args.use_mca\n        and not finetuning_args.use_megatron_bridge\n        and training_args.fp8_enable_fsdp_float8_all_gather\n        and not training_args.fp8\n    ):","sourceCodeStart":512,"sourceCodeEnd":548,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/parser.py#L512-L548","documentation":"Raised in parser.py:530 when model_args.infer_backend is not EngineName.HF (i.e. vllm or sglang) during train/chat argument parsing. The vLLM/SGLang inference engines are only wired into the API server, CLI chat, and WebUI; the training pipeline always runs models through HuggingFace transformers.","triggerScenarios":"A train (or api-less) config/job that sets `infer_backend: vllm` or `infer_backend: sglang` while going through the train flow, so get_train_args sees a non-HF infer_backend.","commonSituations":"Sharing one YAML between serving and training sections; copying an api_server/vllm template into a train config; explicitly setting infer_backend globally in a base config that the train job inherits.","solutions":["Remove `infer_backend` from the training config (it defaults to hf)","If you want vLLM/SGLang serving, use `llamafactory-cli api`/`webui`/`webchat` where those backends are valid","For vLLM-based LoRA evaluation/training pipelines, follow the vllm-based workflows instead of setting infer_backend in the train config"],"exampleFix":"# before (YAML, train job)\ninfer_backend: vllm\ndo_train: true\n\n# after\ndo_train: true\n# infer_backend removed (defaults to hf)","handlingStrategy":"validation","validationCode":"if job_kind == \"train\" and config.get(\"infer_backend\") not in (None, \"hf\"):\n    raise SystemExit(\"infer_backend vllm/sglang belongs to api/cli/web jobs only\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep serving and training YAMLs in separate directories","Set infer_backend only in api/webui job templates"],"tags":["vllm","sglang","infer-backend","training","config-validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}