sgl-project/sglang · error · ImportError

ModelOpt is not available. Please install modelopt.

Error message

ModelOpt is not available. Please install modelopt.

What it means

SGLang's ModelOpt quantization path tries to import modelopt.torch.opt, modelopt.torch.quantization, and is_quantized at runtime; if the modelopt package is not installed the ImportError is re-raised with this message. It means the server was launched with a ModelOpt load format/quantization option but the required NVIDIA modelopt dependency is absent from the environment.

Source

Thrown at python/sglang/srt/model_loader/loader.py:3760

        Args:
            model: The model to quantize
            tokenizer: The tokenizer associated with the model
            quant_cfg: The quantization configuration
            quantized_ckpt_restore_path: Path to restore quantized checkpoint from
            quantized_ckpt_save_path: Path to save quantized checkpoint to
            export_path: Path to export the quantized model in HuggingFace format

        Raises:
            ImportError: If ModelOpt is not available
            Exception: If quantization setup fails
        """
        try:
            import modelopt.torch.opt as mto
            import modelopt.torch.quantization as mtq
            from modelopt.torch.quantization.utils import is_quantized
        except ImportError as e:
            raise ImportError(
                "ModelOpt is not available. Please install modelopt."
            ) from e

        if is_quantized(model):
            rank0_log("Model is already quantized, skipping quantization setup.")
            return
        # Restore from checkpoint if provided
        if quantized_ckpt_restore_path:
            try:
                mto.restore(model, quantized_ckpt_restore_path)
                rank0_log(
                    f"Restored quantized model from {quantized_ckpt_restore_path}"
                )

                # Export model if path provided (even when restoring from checkpoint)
                self._maybe_export_modelopt(model, export_path)
                return
            except Exception as e:

View on GitHub (pinned to 0132848349)

Solutions

1. pip install nvidia-modelopt (or modelopt) matching your torch/CUDA version, then retry
2. Verify the import works: python -c "import modelopt.torch.quantization as mtq; print(mtq.__version__)"
3. If you did not intend ModelOpt quantization, drop the --load-format modelopt / modelopt quantization flags and relaunch
4. Rebuild/use an sglang Docker image that ships modelopt (e.g. the modelopt-tagged images)

Example fix

# before
python -m sglang.launch_server --model meta-llama/Llama-3-8B --load-format modelopt
# ImportError: ModelOpt is not available

# after
pip install nvidia-modelopt
python -m sglang.launch_server --model meta-llama/Llama-3-8B --load-format modelopt

import importlib.util
spec = importlib.util.find_spec("modelopt.torch.quantization")
if spec is None:
    raise SystemExit("Install nvidia-modelopt before using --load-format modelopt")

try:
    launcher.run(args)
except ImportError as e:
    if "ModelOpt" in str(e):
        print("Run: pip install nvidia-modelopt"); raise

When it happens

Trigger: Launching sglang with --load-format modelopt (or a modelopt quantized checkpoint / --modelopt-quant-method style flag) on a machine where `import modelopt` fails, e.g. `pip list | grep modelopt` returns nothing or a broken/incomplete install.

Common situations: Using a Docker image or venv built without modelopt; installing only `modelopt-onnx` or a stale `nvidia-modelopt` version that lacks `modelopt.torch.quantization`; typos in the load format causing fallback into the ModelOpt loader; missing optional extras when installing sglang.

Related errors

• quantize_and_serve requires ModelOpt quantization (set with
• quantize_and_serve functionality is currently disabled due t
• Please install mooncake by following the instructions at htt
• gRPC mode requires the smg-grpc-servicer package. If not ins
• Failed to set up ModelOpt quantization: {e}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ec55891ddb4c9d84. Report an issue: GitHub.