{"record":{"id":"64439dad9372dc88","repo":"hiyouga/LlamaFactory","slug":"vllm-not-install-you-may-need-to-run-pip-install","errorCode":null,"errorMessage":"vLLM not install, you may need to run `pip install vllm`\nor try to use HuggingFace backend: --infer_backend huggingface","messagePattern":"vLLM not install, you may need to run `pip install vllm`\nor try to use HuggingFace backend: --infer_backend huggingface","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/chat/chat_model.py","lineNumber":60,"sourceCode":"    Supports both sync and async methods.\n    Sync methods: chat(), stream_chat() and get_scores().\n    Async methods: achat(), astream_chat() and aget_scores().\n    \"\"\"\n\n    def __init__(self, args: Optional[dict[str, Any]] = None) -> None:\n        model_args, data_args, finetuning_args, generating_args = get_infer_args(args)\n\n        if model_args.infer_backend == EngineName.HF:\n            from .hf_engine import HuggingfaceEngine\n\n            self.engine: BaseEngine = HuggingfaceEngine(model_args, data_args, finetuning_args, generating_args)\n        elif model_args.infer_backend == EngineName.VLLM:\n            try:\n                from .vllm_engine import VllmEngine\n\n                self.engine: BaseEngine = VllmEngine(model_args, data_args, finetuning_args, generating_args)\n            except ImportError as e:\n                raise ImportError(\n                    \"vLLM not install, you may need to run `pip install vllm`\\n\"\n                    \"or try to use HuggingFace backend: --infer_backend huggingface\"\n                ) from e\n        elif model_args.infer_backend == EngineName.SGLANG:\n            try:\n                from .sglang_engine import SGLangEngine\n\n                self.engine: BaseEngine = SGLangEngine(model_args, data_args, finetuning_args, generating_args)\n            except ImportError as e:\n                raise ImportError(\n                    \"SGLang not install, you may need to run `pip install sglang[all]`\\n\"\n                    \"or try to use HuggingFace backend: --infer_backend huggingface\"\n                ) from e\n        else:\n            raise NotImplementedError(f\"Unknown backend: {model_args.infer_backend}\")\n\n        self._loop = asyncio.new_event_loop()\n        self._thread = Thread(target=_start_background_loop, args=(self._loop,), daemon=True)","sourceCodeStart":42,"sourceCodeEnd":78,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/chat/chat_model.py#L42-L78","documentation":"ImportError raised by ChatModel.__init__ when infer_backend is vllm but importing VllmEngine fails. vLLM is an optional dependency; the chained original ImportError is preserved. The message tells you to either install vllm or fall back to the HuggingFace backend.","triggerScenarios":"ChatModel(...) or llamafactory-cli chat/api with --infer_backend vllm on an environment where vllm is absent, partially installed (CUDA mismatch), or fails to import due to a broken dependency chain.","commonSituations":"Installing llamafactory without the [vllm] extra; vllm/torch version conflicts after an upgrade; running on CPU-only machines where vllm cannot import; container images trimmed of GPU libs.","solutions":["Install vllm in the same environment/interpreter (pip install vllm, ideally with the version pinned by this repo's requirements).","Or pass --infer_backend huggingface (infer_backend='huggingface') to use the HF engine.","Diagnose the chained cause: `python -c \"from llamafactory.chat.vllm_engine import VllmEngine\"` to see the underlying ImportError.","Verify torch/CUDA versions match vllm's requirements if the import fails despite installation."],"exampleFix":"# before\nChatModel({'model_name_or_path': ..., 'infer_backend': 'vllm'})\n# after (option 1)\npip install vllm\n# after (option 2)\nChatModel({'model_name_or_path': ..., 'infer_backend': 'huggingface'})","handlingStrategy":"fallback","validationCode":"def vllm_importable():\n    try:\n        import vllm  # noqa: F401\n        return True\n    except ImportError:\n        return False\n\nbackend = \"vllm\" if vllm_importable() else \"huggingface\"","typeGuard":null,"tryCatchPattern":"try { model = ChatModel({..., 'infer_backend': 'vllm'}) } except ImportError as e: if 'vLLM not install' in str(e): model = ChatModel({..., 'infer_backend': 'huggingface'}) else: raise","preventionTips":["Prefer capability detection (importlib.util.find_spec('vllm')) before choosing the backend.","Pin vllm/torch versions consistent with this repo's requirements.","Bake optional backends into the deployment image to avoid runtime surprises."],"tags":["installation","vllm","backend","import-error","inference"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}