hiyouga/LlamaFactory · error · ValueError

vLLM/SGLang backend is only available for API, CLI and Web.

Error message

vLLM/SGLang backend is only available for API, CLI and Web.

What it means

Raised in parser.py:530 when model_args.infer_backend is not EngineName.HF (i.e. vllm or sglang) during train/chat argument parsing. The vLLM/SGLang inference engines are only wired into the API server, CLI chat, and WebUI; the training pipeline always runs models through HuggingFace transformers.

Source

Thrown at src/llamafactory/hparams/parser.py:530

        if finetuning_args.use_badam:
            if finetuning_args.badam_mode == "ratio":
                raise ValueError("Radio-based BAdam does not yet support distributed training, use layer-wise BAdam.")
            elif not is_deepspeed_zero3_enabled():
                raise ValueError("Layer-wise BAdam only supports DeepSpeed ZeRO-3 training.")

    if training_args.deepspeed is not None and (finetuning_args.use_galore or finetuning_args.use_apollo):
        raise ValueError("GaLore and APOLLO are incompatible with DeepSpeed yet.")

    if (
        not finetuning_args.use_mca
        and not finetuning_args.use_megatron_bridge
        and training_args.fp8
        and model_args.quantization_bit is not None
    ):
        raise ValueError("FP8 training is not compatible with quantization. Please disable one of them.")

    if model_args.infer_backend != EngineName.HF:
        raise ValueError("vLLM/SGLang backend is only available for API, CLI and Web.")

    if model_args.use_unsloth and is_deepspeed_zero3_enabled():
        raise ValueError("Unsloth is incompatible with DeepSpeed ZeRO-3.")

    if model_args.use_kt and is_deepspeed_zero3_enabled():
        raise ValueError("KTransformers is incompatible with DeepSpeed ZeRO-3.")

    _set_env_vars()
    _verify_model_args(model_args, data_args, finetuning_args)
    _check_extra_dependencies(model_args, finetuning_args, training_args)
    _verify_trackio_args(training_args)

    if (
        not finetuning_args.use_mca
        and not finetuning_args.use_megatron_bridge
        and training_args.fp8_enable_fsdp_float8_all_gather
        and not training_args.fp8
    ):

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove `infer_backend` from the training config (it defaults to hf)
2. If you want vLLM/SGLang serving, use `llamafactory-cli api`/`webui`/`webchat` where those backends are valid
3. For vLLM-based LoRA evaluation/training pipelines, follow the vllm-based workflows instead of setting infer_backend in the train config

Example fix

# before (YAML, train job)
infer_backend: vllm
do_train: true

# after
do_train: true
# infer_backend removed (defaults to hf)

if job_kind == "train" and config.get("infer_backend") not in (None, "hf"):
    raise SystemExit("infer_backend vllm/sglang belongs to api/cli/web jobs only")

When it happens

Trigger: A train (or api-less) config/job that sets `infer_backend: vllm` or `infer_backend: sglang` while going through the train flow, so get_train_args sees a non-HF infer_backend.

Common situations: Sharing one YAML between serving and training sections; copying an api_server/vllm template into a train config; explicitly setting infer_backend globally in a base config that the train job inherits.

Related errors

• KTransformers inference requires `infer_backend: huggingface
• Please specify dataset for training.
• Cannot use device map for quantized models in training.
• vLLM engine only supports auto-regressive models.
• Unsupported model type: {getattr(config, 'model_type')}.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/a10e036324cfcf14. Report an issue: GitHub.