hiyouga/LlamaFactory · error · ImportError

SGLang not install, you may need to run `pip install sglang[

Error message

SGLang not install, you may need to run `pip install sglang[all]`
or try to use HuggingFace backend: --infer_backend huggingface

What it means

ImportError raised by ChatModel.__init__ when infer_backend is sglang but importing SGLangEngine fails. SGLang is an optional dependency; the original ImportError is chained. The message directs you to install sglang[all] or switch to the HuggingFace backend.

Source

Thrown at src/llamafactory/chat/chat_model.py:70

            self.engine: BaseEngine = HuggingfaceEngine(model_args, data_args, finetuning_args, generating_args)
        elif model_args.infer_backend == EngineName.VLLM:
            try:
                from .vllm_engine import VllmEngine

                self.engine: BaseEngine = VllmEngine(model_args, data_args, finetuning_args, generating_args)
            except ImportError as e:
                raise ImportError(
                    "vLLM not install, you may need to run `pip install vllm`\n"
                    "or try to use HuggingFace backend: --infer_backend huggingface"
                ) from e
        elif model_args.infer_backend == EngineName.SGLANG:
            try:
                from .sglang_engine import SGLangEngine

                self.engine: BaseEngine = SGLangEngine(model_args, data_args, finetuning_args, generating_args)
            except ImportError as e:
                raise ImportError(
                    "SGLang not install, you may need to run `pip install sglang[all]`\n"
                    "or try to use HuggingFace backend: --infer_backend huggingface"
                ) from e
        else:
            raise NotImplementedError(f"Unknown backend: {model_args.infer_backend}")

        self._loop = asyncio.new_event_loop()
        self._thread = Thread(target=_start_background_loop, args=(self._loop,), daemon=True)
        self._thread.start()

    def chat(
        self,
        messages: list[dict[str, str]],
        system: Optional[str] = None,
        tools: Optional[str] = None,
        images: Optional[list["ImageInput"]] = None,
        videos: Optional[list["VideoInput"]] = None,
        audios: Optional[list["AudioInput"]] = None,

View on GitHub (pinned to f28afaf635)

Solutions

1. Install sglang with extras: pip install 'sglang[all]' matching this repo's pinned requirements.
2. Or use --infer_backend huggingface to proceed without sglang.
3. Reproduce the underlying failure: `python -c "from llamafactory.chat.sglang_engine import SGLangEngine"`.
4. Check sglang's CUDA/flashinfer prerequisites on your driver version.

Example fix

# before
ChatModel({...,'infer_backend': 'sglang'})  # sglang not installed
# after
pip install 'sglang[all]'
# or
ChatModel({...,'infer_backend': 'huggingface'})

def sglang_importable():
    try:
        import sglang  # noqa: F401
        return True
    except ImportError:
        return False

backend = "sglang" if sglang_importable() else "huggingface"

try { model = ChatModel({..., 'infer_backend': 'sglang'}) } except ImportError as e: if 'SGLang not install' in str(e): model = ChatModel({..., 'infer_backend': 'huggingface'}) else: raise

When it happens

Trigger: ChatModel(...) / CLI chat with --infer_backend sglang where the sglang package is missing or its import fails (missing CUDA kernels, dependency conflicts).

Common situations: Skipping the sglang extra during install; mixing sglang with an incompatible torch or transformers version; CPU-only environments.

Related errors

• vLLM not install, you may need to run `pip install vllm` or
• SGLang server initialization failed: {str(e)}.
• SGLang only supports n=1.
• SGLang server error: {response.status_code}, {response.text}
• SGLang engine does not support `get_scores`.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/f83272bc76c079dd. Report an issue: GitHub.