{"record":{"id":"ec55891ddb4c9d84","repo":"sgl-project/sglang","slug":"modelopt-is-not-available-please-install-modelopt","errorCode":null,"errorMessage":"ModelOpt is not available. Please install modelopt.","messagePattern":"ModelOpt is not available\\. Please install modelopt\\.","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/model_loader/loader.py","lineNumber":3760,"sourceCode":"\n        Args:\n            model: The model to quantize\n            tokenizer: The tokenizer associated with the model\n            quant_cfg: The quantization configuration\n            quantized_ckpt_restore_path: Path to restore quantized checkpoint from\n            quantized_ckpt_save_path: Path to save quantized checkpoint to\n            export_path: Path to export the quantized model in HuggingFace format\n\n        Raises:\n            ImportError: If ModelOpt is not available\n            Exception: If quantization setup fails\n        \"\"\"\n        try:\n            import modelopt.torch.opt as mto\n            import modelopt.torch.quantization as mtq\n            from modelopt.torch.quantization.utils import is_quantized\n        except ImportError as e:\n            raise ImportError(\n                \"ModelOpt is not available. Please install modelopt.\"\n            ) from e\n\n        if is_quantized(model):\n            rank0_log(\"Model is already quantized, skipping quantization setup.\")\n            return\n        # Restore from checkpoint if provided\n        if quantized_ckpt_restore_path:\n            try:\n                mto.restore(model, quantized_ckpt_restore_path)\n                rank0_log(\n                    f\"Restored quantized model from {quantized_ckpt_restore_path}\"\n                )\n\n                # Export model if path provided (even when restoring from checkpoint)\n                self._maybe_export_modelopt(model, export_path)\n                return\n            except Exception as e:","sourceCodeStart":3742,"sourceCodeEnd":3778,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/model_loader/loader.py#L3742-L3778","documentation":"SGLang's ModelOpt quantization path tries to import modelopt.torch.opt, modelopt.torch.quantization, and is_quantized at runtime; if the modelopt package is not installed the ImportError is re-raised with this message. It means the server was launched with a ModelOpt load format/quantization option but the required NVIDIA modelopt dependency is absent from the environment.","triggerScenarios":"Launching sglang with --load-format modelopt (or a modelopt quantized checkpoint / --modelopt-quant-method style flag) on a machine where `import modelopt` fails, e.g. `pip list | grep modelopt` returns nothing or a broken/incomplete install.","commonSituations":"Using a Docker image or venv built without modelopt; installing only `modelopt-onnx` or a stale `nvidia-modelopt` version that lacks `modelopt.torch.quantization`; typos in the load format causing fallback into the ModelOpt loader; missing optional extras when installing sglang.","solutions":["pip install nvidia-modelopt (or modelopt) matching your torch/CUDA version, then retry","Verify the import works: python -c \"import modelopt.torch.quantization as mtq; print(mtq.__version__)\"","If you did not intend ModelOpt quantization, drop the --load-format modelopt / modelopt quantization flags and relaunch","Rebuild/use an sglang Docker image that ships modelopt (e.g. the modelopt-tagged images)"],"exampleFix":"# before\npython -m sglang.launch_server --model meta-llama/Llama-3-8B --load-format modelopt\n# ImportError: ModelOpt is not available\n\n# after\npip install nvidia-modelopt\npython -m sglang.launch_server --model meta-llama/Llama-3-8B --load-format modelopt","handlingStrategy":"validation","validationCode":"import importlib.util\nspec = importlib.util.find_spec(\"modelopt.torch.quantization\")\nif spec is None:\n    raise SystemExit(\"Install nvidia-modelopt before using --load-format modelopt\")","typeGuard":null,"tryCatchPattern":"try:\n    launcher.run(args)\nexcept ImportError as e:\n    if \"ModelOpt\" in str(e):\n        print(\"Run: pip install nvidia-modelopt\"); raise","preventionTips":["Pre-flight check imports for optional quantization deps before launching the server","Use sglang Docker images that bundle modelopt when doing ModelOpt FP8/FP4 inference","Pin nvidia-modelopt in requirements alongside sglang version"],"tags":["modelopt","quantization","import-error","dependency","sglang"],"backgroundTag":"missing-optional-dependency","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T11:17:15.048Z"}